git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 16/16] reftable/table: move printing logic into test helper

From
Justin Tobler <jltobler@gmail.com>
Date
Apr 2, 2025, 21:52 UTC
Message-ID
<dlrvun53gnrd7lcwjp5m3q6q2gjjavdxnuhj4kie7hp2gfcskb@e7m3mnvk7rbu>
In-Reply-To
<20250331-pks-reftable-polishing-v1-16-ebed5247434c@pks.im>
On 25/03/31 10:41AM, Patrick Steinhardt wrote:
Show 110 quoted lines
> The logic to print individual blocks in a table is hosted in the
> reftable library. This is only the case due to historical reasons though
> because users of the library had no interfaces to read blocks one by
> one. Otherwise, printing individual blocks has no place in the reftable
> library given that the format will not be generic in the first place.
> 
> We have now grown a public interface to iterate through blocks contained
> in a table, and thus we can finally move the logic to print them into
> the test helper.
> 
> Move over the logic and refactor it accordingly. Note that the iterator
> also trivially allows us to access index sections, which we previously
> didn't print at all. This omission wasn't intentional though, so start
> dumping those sections as well so that we can assert that indices are
> written as expected.
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  reftable/reftable-table.h         |  3 --
>  reftable/table.c                  | 65 ------------------------------------
>  t/helper/test-reftable.c          | 69 ++++++++++++++++++++++++++++++++++++++-
>  t/t0613-reftable-write-options.sh |  9 +++++
>  4 files changed, 77 insertions(+), 69 deletions(-)
> 
> diff --git a/reftable/reftable-table.h b/reftable/reftable-table.h
> index f0f1784c664..293fffbddc6 100644
> --- a/reftable/reftable-table.h
> +++ b/reftable/reftable-table.h
> @@ -97,9 +97,6 @@ uint64_t reftable_table_max_update_index(struct reftable_table *t);
>  /* return the min_update_index for a table */
>  uint64_t reftable_table_min_update_index(struct reftable_table *t);
>  
> -/* print blocks onto stdout for debugging. */
> -int reftable_table_print_blocks(const char *tablename);
> -
>  /*
>   * An iterator that iterates through the blocks contained in a given table.
>   */
> diff --git a/reftable/table.c b/reftable/table.c
> index 48f0cdfd42b..8a7581b9800 100644
> --- a/reftable/table.c
> +++ b/reftable/table.c
> @@ -740,71 +740,6 @@ uint64_t reftable_table_min_update_index(struct reftable_table *t)
>  	return t->min_update_index;
>  }
>  
> -int reftable_table_print_blocks(const char *tablename)
> -{
> -	struct {
> -		const char *name;
> -		int type;
> -	} sections[] = {
> -		{
> -			.name = "ref",
> -			.type = REFTABLE_BLOCK_TYPE_REF,
> -		},
> -		{
> -			.name = "obj",
> -			.type = REFTABLE_BLOCK_TYPE_OBJ,
> -		},
> -		{
> -			.name = "log",
> -			.type = REFTABLE_BLOCK_TYPE_LOG,
> -		},
> -	};
> -	struct reftable_block_source src = { 0 };
> -	struct reftable_table *table = NULL;
> -	struct table_iter ti = { 0 };
> -	size_t i;
> -	int err;
> -
> -	err = reftable_block_source_from_file(&src, tablename);
> -	if (err < 0)
> -		goto done;
> -
> -	err = reftable_table_new(&table, &src, tablename);
> -	if (err < 0)
> -		goto done;
> -
> -	table_iter_init(&ti, table);
> -
> -	printf("header:\n");
> -	printf("  block_size: %d\n", table->block_size);
> -
> -	for (i = 0; i < sizeof(sections) / sizeof(*sections); i++) {
> -		err = table_iter_seek_start(&ti, sections[i].type, 0);
> -		if (err < 0)
> -			goto done;
> -		if (err > 0)
> -			continue;
> -
> -		printf("%s:\n", sections[i].name);
> -
> -		while (1) {
> -			printf("  - length: %u\n", ti.block.restart_off);
> -			printf("    restarts: %u\n", ti.block.restart_count);
> -
> -			err = table_iter_next_block(&ti);
> -			if (err < 0)
> -				goto done;
> -			if (err > 0)
> -				break;
> -		}
> -	}
> -
> -done:
> -	reftable_table_decref(table);
> -	table_iter_close(&ti);
> -	return err;
> -}

How that we have a table iterator exposed to iterate over the records we can remove `reftable_table_print_blocks()` from the reftable library. Make sense.

Show 94 quoted lines
>  int reftable_table_init_table_iterator(struct reftable_table *t,
>  				       struct reftable_table_iterator *it)
>  {
> diff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c
> index f8f1956f4f3..c465137826c 100644
> --- a/t/helper/test-reftable.c
> +++ b/t/helper/test-reftable.c
> @@ -2,6 +2,7 @@
>  #include "hash.h"
>  #include "hex.h"
>  #include "reftable/system.h"
> +#include "reftable/reftable-constants.h"
>  #include "reftable/reftable-error.h"
>  #include "reftable/reftable-merged.h"
>  #include "reftable/reftable-stack.h"
> @@ -20,6 +21,72 @@ static void print_help(void)
>  	       "\n");
>  }
>  
> +static int dump_blocks(const char *tablename)
> +{
> +	struct reftable_table_iterator ti = { 0 };
> +	struct reftable_block_source src = { 0 };
> +	struct reftable_table *table = NULL;
> +	uint8_t section_type = 0;
> +	int err;
> +
> +	err = reftable_block_source_from_file(&src, tablename);
> +	if (err < 0)
> +		goto done;
> +
> +	err = reftable_table_new(&table, &src, tablename);
> +	if (err < 0)
> +		goto done;
> +
> +	err = reftable_table_init_table_iterator(table, &ti);
> +	if (err < 0)
> +		goto done;
> +
> +	printf("header:\n");
> +	printf("  block_size: %d\n", table->block_size);
> +
> +	while (1) {
> +		const struct reftable_block *block;
> +
> +		err = reftable_table_iterator_next(&ti, &block);
> +		if (err < 0)
> +			goto done;
> +		if (err > 0)
> +			break;
> +
> +		if (block->block_type != section_type) {
> +			const char *section;
> +			switch (block->block_type) {
> +			case REFTABLE_BLOCK_TYPE_LOG:
> +				section = "log";
> +				break;
> +			case REFTABLE_BLOCK_TYPE_REF:
> +				section = "ref";
> +				break;
> +			case REFTABLE_BLOCK_TYPE_OBJ:
> +				section = "obj";
> +				break;
> +			case REFTABLE_BLOCK_TYPE_INDEX:
> +				section = "idx";
> +				break;
> +			default:
> +				err = -1;
> +				goto done;
> +			}
> +
> +			section_type = block->block_type;
> +			printf("%s:\n", section);
> +		}
> +
> +		printf("  - length: %u\n", block->restart_off);
> +		printf("    restarts: %u\n", block->restart_count);
> +	}
> +
> +done:
> +	reftable_table_iterator_release(&ti);
> +	reftable_table_decref(table);
> +	return err;
> +}
> +
>  static int dump_table(struct reftable_merged_table *mt)
>  {
>  	struct reftable_iterator it = { NULL };
> @@ -184,7 +251,7 @@ int cmd__dump_reftable(int argc, const char **argv)
>  	arg = argv[1];
>  
>  	if (opt_dump_blocks) {
> -		err = reftable_table_print_blocks(arg);
> +		err = dump_blocks(arg);
Nice!
Show 42 quoted lines
>  	} else if (opt_dump_table) {
>  		err = dump_reftable(arg);
>  	} else if (opt_dump_stack) {
> diff --git a/t/t0613-reftable-write-options.sh b/t/t0613-reftable-write-options.sh
> index e2708e11d5b..e4c7461ce9e 100755
> --- a/t/t0613-reftable-write-options.sh
> +++ b/t/t0613-reftable-write-options.sh
> @@ -93,6 +93,9 @@ test_expect_success 'many refs results in multiple blocks' '
>  		    restarts: 3
>  		  - length: 3289
>  		    restarts: 3
> +		idx:
> +		  - length: 103
> +		    restarts: 1
>  		EOF
>  		test-tool dump-reftable -b .git/reftable/*.ref >actual &&
>  		test_cmp expect actual
> @@ -241,6 +244,9 @@ test_expect_success 'object index gets written by default with ref index' '
>  		    restarts: 1
>  		  - length: 80
>  		    restarts: 1
> +		idx:
> +		  - length: 55
> +		    restarts: 2
>  		obj:
>  		  - length: 11
>  		    restarts: 1
> @@ -277,6 +283,9 @@ test_expect_success 'object index can be disabled' '
>  		    restarts: 1
>  		  - length: 80
>  		    restarts: 1
> +		idx:
> +		  - length: 55
> +		    restarts: 2
>  		EOF
>  		test-tool dump-reftable -b .git/reftable/*.ref >actual &&
>  		test_cmp expect actual
> 
> -- 
> 2.49.0.604.gff1f9ca942.dirty
> 
> 
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 37 of 55 in “reftable: overhaul the API to expose access to blocks”
  1. 00/16 reftable: overhaul the API to expose access to blocksPatrick Steinhardt, Mar 31, 2025
  2. 01/16 reftable: fix formatting of the license headerPatrick Steinhardt, Mar 31, 2025
  3. 03/16 reftable/blocksource: consolidate code into a single filePatrick Steinhardt, Mar 31, 2025
  4. Justin ToblerApr 2, 2025
  5. Karthik NayakApr 3, 2025
  6. 02/16 reftable/reader: rename data structure to "table"Patrick Steinhardt, Mar 31, 2025
  7. 04/16 reftable/block: simplify how we track restart pointsPatrick Steinhardt, Mar 31, 2025
  8. Justin ToblerApr 2, 2025
  9. Karthik NayakApr 3, 2025
  10. Patrick SteinhardtApr 7, 2025
  11. 05/16 reftable/table: move reading block into block readerPatrick Steinhardt, Mar 31, 2025
  12. Justin ToblerApr 2, 2025
  13. Patrick SteinhardtApr 7, 2025
  14. 06/16 reftable/block: rename `block` to `block_data`Patrick Steinhardt, Mar 31, 2025
  15. Justin ToblerApr 2, 2025
  16. Patrick SteinhardtApr 7, 2025
  17. 07/16 reftable/block: rename `block_reader` to `reftable_block`Patrick Steinhardt, Mar 31, 2025
  18. Justin ToblerApr 2, 2025
  19. Patrick SteinhardtApr 7, 2025
  20. 08/16 git-zlib: use `struct z_stream_s` instead of typedefPatrick Steinhardt, Mar 31, 2025
  21. 10/16 reftable/block: store block pointer in the block iteratorPatrick Steinhardt, Mar 31, 2025
  22. Justin ToblerApr 2, 2025
  23. Patrick SteinhardtApr 7, 2025
  24. 09/16 reftable/block: create public interface for reading blocksPatrick Steinhardt, Mar 31, 2025
  25. 11/16 reftable/block: make block iterators reseekablePatrick Steinhardt, Mar 31, 2025
  26. Justin ToblerApr 2, 2025
  27. Patrick SteinhardtApr 7, 2025
  28. 12/16 reftable/block: expose a generic iterator over reftable recordsPatrick Steinhardt, Mar 31, 2025
  29. 13/16 reftable/table: add `reftable_table` to the public interfacePatrick Steinhardt, Mar 31, 2025
  30. 14/16 reftable/table: introduce iterator for table blocksPatrick Steinhardt, Mar 31, 2025
  31. Junio C HamanoApr 1, 2025
  32. Patrick SteinhardtApr 2, 2025
  33. Justin ToblerApr 2, 2025
  34. Patrick SteinhardtApr 7, 2025
  35. 15/16 reftable/constants: make block types part of the public interfacePatrick Steinhardt, Mar 31, 2025
  36. 16/16 reftable/table: move printing logic into test helperPatrick Steinhardt, Mar 31, 2025
  37. Justin ToblerApr 2, 2025
  38. 00/16 reftable: overhaul the API to expose access to blocksPatrick Steinhardt, Apr 7, 2025
  39. 01/16 reftable: fix formatting of the license headerPatrick Steinhardt, Apr 7, 2025
  40. 03/16 reftable/blocksource: consolidate code into a single filePatrick Steinhardt, Apr 7, 2025
  41. 02/16 reftable/reader: rename data structure to "table"Patrick Steinhardt, Apr 7, 2025
  42. 04/16 reftable/block: simplify how we track restart pointsPatrick Steinhardt, Apr 7, 2025
  43. 06/16 reftable/block: rename `block` to `block_data`Patrick Steinhardt, Apr 7, 2025
  44. 05/16 reftable/table: move reading block into block readerPatrick Steinhardt, Apr 7, 2025
  45. 07/16 reftable/block: rename `block_reader` to `reftable_block`Patrick Steinhardt, Apr 7, 2025
  46. 08/16 git-zlib: use `struct z_stream_s` instead of typedefPatrick Steinhardt, Apr 7, 2025
  47. 09/16 reftable/block: create public interface for reading blocksPatrick Steinhardt, Apr 7, 2025
  48. 10/16 reftable/block: store block pointer in the block iteratorPatrick Steinhardt, Apr 7, 2025
  49. 11/16 reftable/block: make block iterators reseekablePatrick Steinhardt, Apr 7, 2025
  50. 12/16 reftable/block: expose a generic iterator over reftable recordsPatrick Steinhardt, Apr 7, 2025
  51. 13/16 reftable/table: add `reftable_table` to the public interfacePatrick Steinhardt, Apr 7, 2025
  52. 14/16 reftable/table: introduce iterator for table blocksPatrick Steinhardt, Apr 7, 2025
  53. 15/16 reftable/constants: make block types part of the public interfacePatrick Steinhardt, Apr 7, 2025
  54. 16/16 reftable/table: move printing logic into test helperPatrick Steinhardt, Apr 7, 2025
  55. Justin ToblerApr 14, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.