{"thread":{"id":"63222","subject":"[PATCH 00/16] reftable: overhaul the API to expose access to blocks","startedAt":"2025-03-31T08:41:24Z","lastAt":"2025-04-14T19:46:40Z","messageCount":55,"participants":["Patrick Steinhardt","Junio C Hamano","Justin Tobler","Karthik Nayak"],"isPatch":true,"patchVersion":1,"patchTotal":16},"messages":[{"id":"515338","messageId":"20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im","threadId":"63222","inReplyTo":null,"subject":"[PATCH 00/16] reftable: overhaul the API to expose access to blocks","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-31T08:41:18Z","receivedAt":"2025-03-31T08:41:24Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nthis patch series is a bigger overhaul of the reftable API. This\noverhaul has two main motivations:\n\n  - The reftable library is now standalone and can be used by code bases\n    other than Git, like libgit2. This patch series thus renames a\n    couple of subsystems to have more intuitive names before we gain any\n    new users.\n\n  - Some of the data of reftables isn't accessible at all via public\n    interfaces. Most importantly, it is not possible to access\n    individual blocks in a table. While users shouldn't need that access\n    most of the time, an upcoming usecase that Git itself has is to\n    implement consistency checks for the reftable backend. Here we'll\n    want to read through blocks and their respective records one by one\n    to ensure that they are sane and then iterate through records\n    contained in these blocks.\n\nThe patch series is structured as follows:\n\n  - Patch 1 is a trivial global refactoring to fix formatting of the\n    license headers. They have been annoying me for far too long.\n\n  - Patches 2 to 8 consolidate and rename a couple of data structures:\n\n      - `reftable_reader` becomes `reftable_table`, as it is used to\n        access an individual table.\n\n      - `reftable_block` becomes `reftable_block_data`, as it is only a\n        simple container for the underlying bytes.\n\n      - `reftable_block_reader` becomes `reftable_block`, as it is used\n        to access an individual block.\n\n    Overall, the data structures are now called after what they provide\n    access to compared to the rather generic previous names. This is\n    also in line with other data structures like `reftable_merged_table`\n    and `reftable_stack`.\n\n  - Patches 9 to 13 refactor the block interface so that it can expose a\n    generic `reftable_iterator`, granting generic access to all of its\n    contained records.\n\n  - Patches 14 to 16 refactor the table interface to expose a new\n    iterator over its contained blocks.\n\n  - Patch 17 refactors `reftable_table_print_blocks` to be implemented\n    on top of these new iterators. This allows us to move it out of the\n    library codebase into the test helper.\n\nThe series is built on Git v2.49.0 with ps/reftable-sans-compat-util at\n8f6a2dbe340 (Makefile: skip reftable library for Coccinelle, 2025-02-18)\nmerged into it.\n\nThanks!\n\nPatrick\n\n---\nPatrick Steinhardt (16):\n      reftable: fix formatting of the license header\n      reftable/reader: rename data structure to \"table\"\n      reftable/blocksource: consolidate code into a single file\n      reftable/block: simplify how we track restart points\n      reftable/table: move reading block into block reader\n      reftable/block: rename `block` to `block_data`\n      reftable/block: rename `block_reader` to `reftable_block`\n      git-zlib: use `struct z_stream_s` instead of typedef\n      reftable/block: create public interface for reading blocks\n      reftable/block: store block pointer in the block iterator\n      reftable/block: make block iterators reseekable\n      reftable/block: expose a generic iterator over reftable records\n      reftable/table: add `reftable_table` to the public interface\n      reftable/table: introduce iterator for table blocks\n      reftable/constants: make block types part of the public interface\n      reftable/table: move printing logic into test helper\n\n .../howto/recover-corrupted-object-harder.adoc     |   4 +-\n Makefile                                           |   4 +-\n compat/zlib-compat.h                               |   4 +-\n git-zlib.h                                         |   2 +-\n meson.build                                        |   2 +-\n reftable/basics.c                                  |  12 +-\n reftable/basics.h                                  |  19 +-\n reftable/block.c                                   | 284 ++++++++-----\n reftable/block.h                                   |  85 ++--\n reftable/blocksource.c                             |  67 ++-\n reftable/blocksource.h                             |  39 +-\n reftable/constants.h                               |  18 +-\n reftable/error.c                                   |  12 +-\n reftable/iter.c                                    |  36 +-\n reftable/iter.h                                    |  18 +-\n reftable/merged.c                                  |  42 +-\n reftable/merged.h                                  |  16 +-\n reftable/pq.c                                      |  12 +-\n reftable/pq.h                                      |  12 +-\n reftable/reader.h                                  |  67 ---\n reftable/record.c                                  |  52 +--\n reftable/record.h                                  |  12 +-\n reftable/reftable-basics.h                         |  10 +-\n reftable/reftable-block.h                          |  74 ++++\n reftable/reftable-blocksource.h                    |  29 +-\n reftable/reftable-constants.h                      |  18 +\n reftable/reftable-error.h                          |  12 +-\n reftable/reftable-iterator.h                       |  12 +-\n reftable/reftable-merged.h                         |  18 +-\n reftable/reftable-reader.h                         |  72 ----\n reftable/reftable-record.h                         |  12 +-\n reftable/reftable-stack.h                          |  12 +-\n reftable/reftable-table.h                          | 115 +++++\n reftable/reftable-writer.h                         |  12 +-\n reftable/stack.c                                   | 188 ++++-----\n reftable/stack.h                                   |  16 +-\n reftable/system.h                                  |  12 +-\n reftable/{reader.c => table.c}                     | 463 +++++++++------------\n reftable/table.h                                   |  29 ++\n reftable/tree.c                                    |  12 +-\n reftable/tree.h                                    |  12 +-\n reftable/writer.c                                  |  34 +-\n reftable/writer.h                                  |  12 +-\n t/helper/test-reftable.c                           |  81 +++-\n t/meson.build                                      |   2 +-\n t/t0613-reftable-write-options.sh                  |   9 +\n t/unit-tests/t-reftable-block.c                    | 218 +++++++---\n t/unit-tests/t-reftable-merged.c                   |  86 ++--\n t/unit-tests/t-reftable-pq.c                       |  10 +-\n t/unit-tests/t-reftable-reader.c                   |  96 -----\n t/unit-tests/t-reftable-readwrite.c                | 106 ++---\n t/unit-tests/t-reftable-record.c                   |  40 +-\n t/unit-tests/t-reftable-stack.c                    |  66 +--\n t/unit-tests/t-reftable-table.c                    | 205 +++++++++\n 54 files changed, 1638 insertions(+), 1274 deletions(-)\n\n\n---\nbase-commit: bc705c20b9a88abaff0f379bab6d545f012656af\nchange-id: 20241210-pks-reftable-polishing-332ce318cdea\n\n"},{"id":"515339","messageId":"20250331-pks-reftable-polishing-v1-1-ebed5247434c@pks.im","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im","subject":"[PATCH 01/16] reftable: fix formatting of the license header","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-31T08:41:19Z","receivedAt":"2025-03-31T08:41:25Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The license headers used across the reftable library doesn't follow our\ntypical coding style for multi-line comments. Fix it.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/basics.c               | 12 ++++++------\n reftable/basics.h               | 12 ++++++------\n reftable/block.c                | 12 ++++++------\n reftable/block.h                | 12 ++++++------\n reftable/blocksource.c          | 12 ++++++------\n reftable/blocksource.h          | 12 ++++++------\n reftable/constants.h            | 12 ++++++------\n reftable/error.c                | 12 ++++++------\n reftable/iter.c                 | 12 ++++++------\n reftable/iter.h                 | 12 ++++++------\n reftable/merged.c               | 12 ++++++------\n reftable/merged.h               | 12 ++++++------\n reftable/pq.c                   | 12 ++++++------\n reftable/pq.h                   | 12 ++++++------\n reftable/reader.c               | 12 ++++++------\n reftable/reader.h               | 12 ++++++------\n reftable/record.c               | 12 ++++++------\n reftable/record.h               | 12 ++++++------\n reftable/reftable-basics.h      |  2 +-\n reftable/reftable-blocksource.h | 12 ++++++------\n reftable/reftable-error.h       | 12 ++++++------\n reftable/reftable-iterator.h    | 12 ++++++------\n reftable/reftable-merged.h      | 12 ++++++------\n reftable/reftable-reader.h      | 12 ++++++------\n reftable/reftable-record.h      | 12 ++++++------\n reftable/reftable-stack.h       | 12 ++++++------\n reftable/reftable-writer.h      | 12 ++++++------\n reftable/stack.c                | 12 ++++++------\n reftable/stack.h                | 12 ++++++------\n reftable/system.h               | 12 ++++++------\n reftable/tree.c                 | 12 ++++++------\n reftable/tree.h                 | 12 ++++++------\n reftable/writer.c               | 12 ++++++------\n reftable/writer.h               | 12 ++++++------\n 34 files changed, 199 insertions(+), 199 deletions(-)\n\ndiff --git a/reftable/basics.c b/reftable/basics.c\nindex 8c4a4433e41..9988ebd635e 100644\n--- a/reftable/basics.c\n+++ b/reftable/basics.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #define REFTABLE_ALLOW_BANNED_ALLOCATORS\n #include \"basics.h\"\ndiff --git a/reftable/basics.h b/reftable/basics.h\nindex fd59cbb7720..96a2f0d3829 100644\n--- a/reftable/basics.h\n+++ b/reftable/basics.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef BASICS_H\n #define BASICS_H\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 53b5e044690..a5734d44415 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"block.h\"\n \ndiff --git a/reftable/block.h b/reftable/block.h\nindex bef2b8a4c5c..eaeffdffc90 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef BLOCK_H\n #define BLOCK_H\ndiff --git a/reftable/blocksource.c b/reftable/blocksource.c\nindex 78c1be23373..1397cbe7800 100644\n--- a/reftable/blocksource.c\n+++ b/reftable/blocksource.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"system.h\"\n \ndiff --git a/reftable/blocksource.h b/reftable/blocksource.h\nindex a84a3ccd891..7b67898ae22 100644\n--- a/reftable/blocksource.h\n+++ b/reftable/blocksource.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef BLOCKSOURCE_H\n #define BLOCKSOURCE_H\ndiff --git a/reftable/constants.h b/reftable/constants.h\nindex f6beb843ebf..091728cf033 100644\n--- a/reftable/constants.h\n+++ b/reftable/constants.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef CONSTANTS_H\n #define CONSTANTS_H\ndiff --git a/reftable/error.c b/reftable/error.c\nindex 660d0296170..c7cab2dbc42 100644\n--- a/reftable/error.c\n+++ b/reftable/error.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"system.h\"\n #include \"reftable-error.h\"\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex f520382e700..44a155e756e 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"iter.h\"\n \ndiff --git a/reftable/iter.h b/reftable/iter.h\nindex 40f98893b85..bd217405f55 100644\n--- a/reftable/iter.h\n+++ b/reftable/iter.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef ITER_H\n #define ITER_H\ndiff --git a/reftable/merged.c b/reftable/merged.c\nindex 4ff1553772a..1829a081756 100644\n--- a/reftable/merged.c\n+++ b/reftable/merged.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"merged.h\"\n \ndiff --git a/reftable/merged.h b/reftable/merged.h\nindex 0b7d939e92b..ae92409d342 100644\n--- a/reftable/merged.h\n+++ b/reftable/merged.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef MERGED_H\n #define MERGED_H\ndiff --git a/reftable/pq.c b/reftable/pq.c\nindex 82394a972db..9a79f5c5eec 100644\n--- a/reftable/pq.c\n+++ b/reftable/pq.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"pq.h\"\n \ndiff --git a/reftable/pq.h b/reftable/pq.h\nindex ff39016445b..42310670b09 100644\n--- a/reftable/pq.h\n+++ b/reftable/pq.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef PQ_H\n #define PQ_H\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 172aff2c10b..155a26c30ff 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"reader.h\"\n \ndiff --git a/reftable/reader.h b/reftable/reader.h\nindex bb72108a6f1..c9dccbd6c97 100644\n--- a/reftable/reader.h\n+++ b/reftable/reader.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef READER_H\n #define READER_H\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 142853d5070..26cd834d405 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n /* record.c - methods for different types of records. */\n \ndiff --git a/reftable/record.h b/reftable/record.h\nindex 867810a9328..7953f352a3c 100644\n--- a/reftable/record.h\n+++ b/reftable/record.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef RECORD_H\n #define RECORD_H\ndiff --git a/reftable/reftable-basics.h b/reftable/reftable-basics.h\nindex e0397ed5836..ed7c7c9ac2c 100644\n--- a/reftable/reftable-basics.h\n+++ b/reftable/reftable-basics.h\n@@ -4,7 +4,7 @@\n  * Use of this source code is governed by a BSD-style\n  * license that can be found in the LICENSE file or at\n  * https://developers.google.com/open-source/licenses/bsd\n-*/\n+ */\n \n #ifndef REFTABLE_BASICS_H\n #define REFTABLE_BASICS_H\ndiff --git a/reftable/reftable-blocksource.h b/reftable/reftable-blocksource.h\nindex 6b326aa5ea5..8692cd017e9 100644\n--- a/reftable/reftable-blocksource.h\n+++ b/reftable/reftable-blocksource.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef REFTABLE_BLOCKSOURCE_H\n #define REFTABLE_BLOCKSOURCE_H\ndiff --git a/reftable/reftable-error.h b/reftable/reftable-error.h\nindex a7e33d964d0..d100e0df927 100644\n--- a/reftable/reftable-error.h\n+++ b/reftable/reftable-error.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef REFTABLE_ERROR_H\n #define REFTABLE_ERROR_H\ndiff --git a/reftable/reftable-iterator.h b/reftable/reftable-iterator.h\nindex e3bf688d53d..af582028c27 100644\n--- a/reftable/reftable-iterator.h\n+++ b/reftable/reftable-iterator.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef REFTABLE_ITERATOR_H\n #define REFTABLE_ITERATOR_H\ndiff --git a/reftable/reftable-merged.h b/reftable/reftable-merged.h\nindex f2d01c3ef82..445756e475a 100644\n--- a/reftable/reftable-merged.h\n+++ b/reftable/reftable-merged.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef REFTABLE_MERGED_H\n #define REFTABLE_MERGED_H\ndiff --git a/reftable/reftable-reader.h b/reftable/reftable-reader.h\nindex 0085fbb9032..9b6a8ac0716 100644\n--- a/reftable/reftable-reader.h\n+++ b/reftable/reftable-reader.h\n@@ -1,10 +1,10 @@\n /*\n-  Copyright 2020 Google LLC\n-\n-  Use of this source code is governed by a BSD-style\n-  license that can be found in the LICENSE file or at\n-  https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef REFTABLE_READER_H\n #define REFTABLE_READER_H\ndiff --git a/reftable/reftable-record.h b/reftable/reftable-record.h\nindex 931e5947441..385a74cc864 100644\n--- a/reftable/reftable-record.h\n+++ b/reftable/reftable-record.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef REFTABLE_RECORD_H\n #define REFTABLE_RECORD_H\ndiff --git a/reftable/reftable-stack.h b/reftable/reftable-stack.h\nindex ae14270ea74..910ec6ef3a2 100644\n--- a/reftable/reftable-stack.h\n+++ b/reftable/reftable-stack.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef REFTABLE_STACK_H\n #define REFTABLE_STACK_H\ndiff --git a/reftable/reftable-writer.h b/reftable/reftable-writer.h\nindex 1befe3b07cf..0fbeff17f46 100644\n--- a/reftable/reftable-writer.h\n+++ b/reftable/reftable-writer.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef REFTABLE_WRITER_H\n #define REFTABLE_WRITER_H\ndiff --git a/reftable/stack.c b/reftable/stack.c\nindex 6dac015b473..bc3cfa91170 100644\n--- a/reftable/stack.c\n+++ b/reftable/stack.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"stack.h\"\n \ndiff --git a/reftable/stack.h b/reftable/stack.h\nindex 5b45cff4f75..d91f5ce6072 100644\n--- a/reftable/stack.h\n+++ b/reftable/stack.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef STACK_H\n #define STACK_H\ndiff --git a/reftable/system.h b/reftable/system.h\nindex 10055fbff2d..cf207aa0c53 100644\n--- a/reftable/system.h\n+++ b/reftable/system.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef SYSTEM_H\n #define SYSTEM_H\ndiff --git a/reftable/tree.c b/reftable/tree.c\nindex f4dbe720901..a52f7c0c7d4 100644\n--- a/reftable/tree.c\n+++ b/reftable/tree.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"system.h\"\n #include \"tree.h\"\ndiff --git a/reftable/tree.h b/reftable/tree.h\nindex 9604453b6d5..2c9c4652994 100644\n--- a/reftable/tree.h\n+++ b/reftable/tree.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef TREE_H\n #define TREE_H\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex ce55a1deb06..f0accfd0c32 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"writer.h\"\n \ndiff --git a/reftable/writer.h b/reftable/writer.h\nindex 1f4788a430c..9f53610b27a 100644\n--- a/reftable/writer.h\n+++ b/reftable/writer.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef WRITER_H\n #define WRITER_H\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515340","messageId":"20250331-pks-reftable-polishing-v1-3-ebed5247434c@pks.im","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im","subject":"[PATCH 03/16] reftable/blocksource: consolidate code into a single file","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-31T08:41:21Z","receivedAt":"2025-03-31T08:41:25Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The code that implements block sources is distributed across a couple of\nfiles even though. Consolidate all of it into \"reftable/blocksource.c\"\nand its accompanying header so that it is easier to locate and more self\ncontained.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/block.c                    | 17 +++--------------\n reftable/block.h                    |  3 ---\n reftable/blocksource.c              | 35 +++++++++++++++++++++++++++++++++++\n reftable/blocksource.h              | 27 ++++++++++++++++++++++++++-\n reftable/iter.c                     |  5 +++--\n reftable/reftable-blocksource.h     |  3 ++-\n reftable/table.c                    | 33 +++++----------------------------\n reftable/table.h                    |  7 -------\n t/unit-tests/t-reftable-block.c     |  8 ++++----\n t/unit-tests/t-reftable-readwrite.c |  4 ++--\n 10 files changed, 80 insertions(+), 62 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex a5734d44415..97740187259 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -221,7 +221,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tuint32_t restart_start = 0;\n \tuint8_t *restart_bytes = NULL;\n \n-\treftable_block_done(&br->block);\n+\tblock_source_return_block(&br->block);\n \n \tif (!reftable_is_block_type(typ)) {\n \t\terr =  REFTABLE_FORMAT_ERROR;\n@@ -285,7 +285,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\t}\n \n \t\t/* We're done with the input data. */\n-\t\treftable_block_done(block);\n+\t\tblock_source_return_block(block);\n \t\tblock->data = br->uncompressed_data;\n \t\tblock->len = sz;\n \t\tfull_block_size = src_len + block_header_skip - br->zstream->avail_in;\n@@ -324,7 +324,7 @@ void block_reader_release(struct block_reader *br)\n \tinflateEnd(br->zstream);\n \treftable_free(br->zstream);\n \treftable_free(br->uncompressed_data);\n-\treftable_block_done(&br->block);\n+\tblock_source_return_block(&br->block);\n }\n \n uint8_t block_reader_type(const struct block_reader *r)\n@@ -570,14 +570,3 @@ void block_writer_release(struct block_writer *bw)\n \treftable_buf_release(&bw->last_key);\n \t/* the block is not owned. */\n }\n-\n-void reftable_block_done(struct reftable_block *blockp)\n-{\n-\tstruct reftable_block_source source = blockp->source;\n-\tif (blockp && source.ops)\n-\t\tsource.ops->return_block(source.arg, blockp);\n-\tblockp->data = NULL;\n-\tblockp->len = 0;\n-\tblockp->source.ops = NULL;\n-\tblockp->source.arg = NULL;\n-}\ndiff --git a/reftable/block.h b/reftable/block.h\nindex eaeffdffc90..203b07d9a44 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -142,7 +142,4 @@ size_t header_size(int version);\n /* size of file footer, depending on format version */\n size_t footer_size(int version);\n \n-/* returns a block to its source. */\n-void reftable_block_done(struct reftable_block *ret);\n-\n #endif\ndiff --git a/reftable/blocksource.c b/reftable/blocksource.c\nindex 1397cbe7800..bc785506fb1 100644\n--- a/reftable/blocksource.c\n+++ b/reftable/blocksource.c\n@@ -13,6 +13,41 @@\n #include \"reftable-blocksource.h\"\n #include \"reftable-error.h\"\n \n+void block_source_return_block(struct reftable_block *block)\n+{\n+\tstruct reftable_block_source source = block->source;\n+\tif (block && source.ops)\n+\t\tsource.ops->return_block(source.arg, block);\n+\tblock->data = NULL;\n+\tblock->len = 0;\n+\tblock->source.ops = NULL;\n+\tblock->source.arg = NULL;\n+}\n+\n+void block_source_close(struct reftable_block_source *source)\n+{\n+\tif (!source->ops) {\n+\t\treturn;\n+\t}\n+\n+\tsource->ops->close(source->arg);\n+\tsource->ops = NULL;\n+}\n+\n+ssize_t block_source_read_block(struct reftable_block_source *source,\n+\t\t\t\tstruct reftable_block *dest, uint64_t off,\n+\t\t\t\tuint32_t size)\n+{\n+\tssize_t result = source->ops->read_block(source->arg, dest, off, size);\n+\tdest->source = *source;\n+\treturn result;\n+}\n+\n+uint64_t block_source_size(struct reftable_block_source *source)\n+{\n+\treturn source->ops->size(source->arg);\n+}\n+\n static void reftable_buf_return_block(void *b REFTABLE_UNUSED, struct reftable_block *dest)\n {\n \tif (dest->len)\ndiff --git a/reftable/blocksource.h b/reftable/blocksource.h\nindex 7b67898ae22..639b9a1a3c5 100644\n--- a/reftable/blocksource.h\n+++ b/reftable/blocksource.h\n@@ -12,9 +12,34 @@\n #include \"system.h\"\n \n struct reftable_block_source;\n+struct reftable_block;\n struct reftable_buf;\n \n-/* Create an in-memory block source for reading reftables */\n+/*\n+ * Close the block source and the underlying resource. This is a no-op in case\n+ * the block source is zero-initialized.\n+ */\n+void block_source_close(struct reftable_block_source *source);\n+\n+/*\n+ * Read a block of length `size` from the source at the given `off`.\n+ */\n+ssize_t block_source_read_block(struct reftable_block_source *source,\n+\t\t\t\tstruct reftable_block *dest, uint64_t off,\n+\t\t\t\tuint32_t size);\n+\n+/*\n+ * Return the total length of the underlying resource.\n+ */\n+uint64_t block_source_size(struct reftable_block_source *source);\n+\n+/*\n+ * Return a block to its original source, releasing any resources associated\n+ * with it.\n+ */\n+void block_source_return_block(struct reftable_block *block);\n+\n+/* Create an in-memory block source for reading reftables. */\n void block_source_from_buf(struct reftable_block_source *bs,\n \t\t\t   struct reftable_buf *buf);\n \ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex 7376f263c99..6af6eb49396 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -11,6 +11,7 @@\n #include \"system.h\"\n \n #include \"block.h\"\n+#include \"blocksource.h\"\n #include \"constants.h\"\n #include \"reftable-error.h\"\n #include \"table.h\"\n@@ -113,7 +114,7 @@ static void indexed_table_ref_iter_close(void *p)\n {\n \tstruct indexed_table_ref_iter *it = p;\n \tblock_iter_close(&it->cur);\n-\treftable_block_done(&it->block_reader.block);\n+\tblock_source_return_block(&it->block_reader.block);\n \treftable_free(it->offsets);\n \treftable_buf_release(&it->oid);\n }\n@@ -127,7 +128,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n \t\treturn 1;\n \t}\n \n-\treftable_block_done(&it->block_reader.block);\n+\tblock_source_return_block(&it->block_reader.block);\n \n \toff = it->offsets[it->offset_idx++];\n \terr = table_init_block_reader(it->table, &it->block_reader, off,\ndiff --git a/reftable/reftable-blocksource.h b/reftable/reftable-blocksource.h\nindex 8692cd017e9..96430b629e4 100644\n--- a/reftable/reftable-blocksource.h\n+++ b/reftable/reftable-blocksource.h\n@@ -11,7 +11,8 @@\n \n #include <stdint.h>\n \n-/* block_source is a generic wrapper for a seekable readable file.\n+/*\n+ * Generic wrapper for a seekable readable file.\n  */\n struct reftable_block_source {\n \tstruct reftable_block_source_vtable *ops;\ndiff --git a/reftable/table.c b/reftable/table.c\nindex 440fb559ad1..d18e17b0d44 100644\n--- a/reftable/table.c\n+++ b/reftable/table.c\n@@ -10,35 +10,12 @@\n \n #include \"system.h\"\n #include \"block.h\"\n+#include \"blocksource.h\"\n #include \"constants.h\"\n #include \"iter.h\"\n #include \"record.h\"\n #include \"reftable-error.h\"\n \n-uint64_t block_source_size(struct reftable_block_source *source)\n-{\n-\treturn source->ops->size(source->arg);\n-}\n-\n-ssize_t block_source_read_block(struct reftable_block_source *source,\n-\t\t\t\tstruct reftable_block *dest, uint64_t off,\n-\t\t\t\tuint32_t size)\n-{\n-\tssize_t result = source->ops->read_block(source->arg, dest, off, size);\n-\tdest->source = *source;\n-\treturn result;\n-}\n-\n-void block_source_close(struct reftable_block_source *source)\n-{\n-\tif (!source->ops) {\n-\t\treturn;\n-\t}\n-\n-\tsource->ops->close(source->arg);\n-\tsource->ops = NULL;\n-}\n-\n static struct reftable_table_offsets *\n table_offsets_for(struct reftable_table *t, uint8_t typ)\n {\n@@ -249,7 +226,7 @@ int table_init_block_reader(struct reftable_table *t, struct block_reader *br,\n \t}\n \n \tif (block_size > guess_block_size) {\n-\t\treftable_block_done(&block);\n+\t\tblock_source_return_block(&block);\n \t\terr = table_get_block(t, &block, next_off, block_size);\n \t\tif (err < 0) {\n \t\t\tgoto done;\n@@ -259,7 +236,7 @@ int table_init_block_reader(struct reftable_table *t, struct block_reader *br,\n \terr = block_reader_init(br, &block, header_off, t->block_size,\n \t\t\t\thash_size(t->hash_id));\n done:\n-\treftable_block_done(&block);\n+\tblock_source_return_block(&block);\n \n \treturn err;\n }\n@@ -666,8 +643,8 @@ int reftable_table_new(struct reftable_table **out,\n \t*out = t;\n \n done:\n-\treftable_block_done(&footer);\n-\treftable_block_done(&header);\n+\tblock_source_return_block(&footer);\n+\tblock_source_return_block(&header);\n \tif (err) {\n \t\tif (t)\n \t\t\treftable_free(t->name);\ndiff --git a/reftable/table.h b/reftable/table.h\nindex 9cd8f80a207..8d8dd2b413d 100644\n--- a/reftable/table.h\n+++ b/reftable/table.h\n@@ -14,13 +14,6 @@\n #include \"reftable-iterator.h\"\n #include \"reftable-table.h\"\n \n-uint64_t block_source_size(struct reftable_block_source *source);\n-\n-ssize_t block_source_read_block(struct reftable_block_source *source,\n-\t\t\t\tstruct reftable_block *dest, uint64_t off,\n-\t\t\t\tuint32_t size);\n-void block_source_close(struct reftable_block_source *source);\n-\n /* metadata for a block type */\n struct reftable_table_offsets {\n \tint is_present;\ndiff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c\nindex 22040aeefa5..8bb40482347 100644\n--- a/t/unit-tests/t-reftable-block.c\n+++ b/t/unit-tests/t-reftable-block.c\n@@ -100,7 +100,7 @@ static void t_ref_block_read_write(void)\n \tblock_reader_release(&br);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n-\treftable_block_done(&br.block);\n+\tblock_source_return_block(&br.block);\n \treftable_buf_release(&want);\n \treftable_buf_release(&buf);\n \tfor (i = 0; i < N; i++)\n@@ -190,7 +190,7 @@ static void t_log_block_read_write(void)\n \tblock_reader_release(&br);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n-\treftable_block_done(&br.block);\n+\tblock_source_return_block(&br.block);\n \treftable_buf_release(&want);\n \treftable_buf_release(&buf);\n \tfor (i = 0; i < N; i++)\n@@ -273,7 +273,7 @@ static void t_obj_block_read_write(void)\n \tblock_reader_release(&br);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n-\treftable_block_done(&br.block);\n+\tblock_source_return_block(&br.block);\n \treftable_buf_release(&want);\n \treftable_buf_release(&buf);\n \tfor (i = 0; i < N; i++)\n@@ -365,7 +365,7 @@ static void t_index_block_read_write(void)\n \tblock_reader_release(&br);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n-\treftable_block_done(&br.block);\n+\tblock_source_return_block(&br.block);\n \treftable_buf_release(&want);\n \treftable_buf_release(&buf);\n \tfor (i = 0; i < N; i++)\ndiff --git a/t/unit-tests/t-reftable-readwrite.c b/t/unit-tests/t-reftable-readwrite.c\nindex c4c27242ba9..3fba888cdaa 100644\n--- a/t/unit-tests/t-reftable-readwrite.c\n+++ b/t/unit-tests/t-reftable-readwrite.c\n@@ -32,13 +32,13 @@ static void t_buffer(void)\n \tn = block_source_read_block(&source, &out, 0, sizeof(in));\n \tcheck_int(n, ==, sizeof(in));\n \tcheck(!memcmp(in, out.data, n));\n-\treftable_block_done(&out);\n+\tblock_source_return_block(&out);\n \n \tn = block_source_read_block(&source, &out, 1, 2);\n \tcheck_int(n, ==, 2);\n \tcheck(!memcmp(out.data, \"el\", 2));\n \n-\treftable_block_done(&out);\n+\tblock_source_return_block(&out);\n \tblock_source_close(&source);\n \treftable_buf_release(&buf);\n }\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515343","messageId":"20250331-pks-reftable-polishing-v1-2-ebed5247434c@pks.im","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im","subject":"[PATCH 02/16] reftable/reader: rename data structure to \"table\"","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-31T08:41:20Z","receivedAt":"2025-03-31T08:41:25Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The `struct reftable_reader` subsystem encapsulates a table that has\nbeen read from the disk. As such, the current name of that structure is\nsomewhat hard to understand as it only talks about the fact that we read\nsomething from disk, without really giving an indicator _what_ that is.\n\nFurthermore, this naming schema doesn't really fit well into how the\nother structures are named: `reftable_merged_table`, `reftable_stack`,\n`reftable_block` and `reftable_record` are all named after what they\nencapsulate.\n\nRename the subsystem to `reftable_table`, which directly gives a hint\nthat the data structure is about handling the individual tables part of\nthe stack.\n\nWhile this change results in a lot of churn, it prepares for us exposing\nthe APIs to third-party callers now that the reftable library is a\nstandalone library that can be linked against by other projects.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n Makefile                                           |   4 +-\n meson.build                                        |   2 +-\n reftable/iter.c                                    |  10 +-\n reftable/iter.h                                    |   4 +-\n reftable/merged.c                                  |  26 +--\n reftable/merged.h                                  |   4 +-\n reftable/reftable-merged.h                         |   6 +-\n reftable/reftable-reader.h                         |  72 ------\n reftable/reftable-table.h                          |  72 ++++++\n reftable/stack.c                                   | 168 +++++++-------\n reftable/stack.h                                   |   4 +-\n reftable/{reader.c => table.c}                     | 254 ++++++++++-----------\n reftable/{reader.h => table.h}                     |  30 +--\n t/helper/test-reftable.c                           |  14 +-\n t/meson.build                                      |   2 +-\n t/unit-tests/t-reftable-merged.c                   |  74 +++---\n t/unit-tests/t-reftable-readwrite.c                |  96 ++++----\n t/unit-tests/t-reftable-stack.c                    |  66 +++---\n .../{t-reftable-reader.c => t-reftable-table.c}    |  26 +--\n 19 files changed, 467 insertions(+), 467 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 22d632da4ea..58bd9dc6452 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1374,10 +1374,10 @@ UNIT_TEST_PROGRAMS += t-reftable-basics\n UNIT_TEST_PROGRAMS += t-reftable-block\n UNIT_TEST_PROGRAMS += t-reftable-merged\n UNIT_TEST_PROGRAMS += t-reftable-pq\n-UNIT_TEST_PROGRAMS += t-reftable-reader\n UNIT_TEST_PROGRAMS += t-reftable-readwrite\n UNIT_TEST_PROGRAMS += t-reftable-record\n UNIT_TEST_PROGRAMS += t-reftable-stack\n+UNIT_TEST_PROGRAMS += t-reftable-table\n UNIT_TEST_PROGRAMS += t-trailer\n UNIT_TEST_PROGRAMS += t-urlmatch-normalization\n UNIT_TEST_PROGS = $(patsubst %,$(UNIT_TEST_BIN)/%$X,$(UNIT_TEST_PROGRAMS))\n@@ -2729,10 +2729,10 @@ REFTABLE_OBJS += reftable/blocksource.o\n REFTABLE_OBJS += reftable/iter.o\n REFTABLE_OBJS += reftable/merged.o\n REFTABLE_OBJS += reftable/pq.o\n-REFTABLE_OBJS += reftable/reader.o\n REFTABLE_OBJS += reftable/record.o\n REFTABLE_OBJS += reftable/stack.o\n REFTABLE_OBJS += reftable/system.o\n+REFTABLE_OBJS += reftable/table.o\n REFTABLE_OBJS += reftable/tree.o\n REFTABLE_OBJS += reftable/writer.o\n \ndiff --git a/meson.build b/meson.build\nindex efe2871c9db..61986cc48db 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -410,10 +410,10 @@ libgit_sources = [\n   'reftable/iter.c',\n   'reftable/merged.c',\n   'reftable/pq.c',\n-  'reftable/reader.c',\n   'reftable/record.c',\n   'reftable/stack.c',\n   'reftable/system.c',\n+  'reftable/table.c',\n   'reftable/tree.c',\n   'reftable/writer.c',\n   'remote.c',\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex 44a155e756e..7376f263c99 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -12,8 +12,8 @@\n \n #include \"block.h\"\n #include \"constants.h\"\n-#include \"reader.h\"\n #include \"reftable-error.h\"\n+#include \"table.h\"\n \n int iterator_seek(struct reftable_iterator *it, struct reftable_record *want)\n {\n@@ -130,8 +130,8 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n \treftable_block_done(&it->block_reader.block);\n \n \toff = it->offsets[it->offset_idx++];\n-\terr = reader_init_block_reader(it->r, &it->block_reader, off,\n-\t\t\t\t       BLOCK_TYPE_REF);\n+\terr = table_init_block_reader(it->table, &it->block_reader, off,\n+\t\t\t\t      BLOCK_TYPE_REF);\n \tif (err < 0) {\n \t\treturn err;\n \t}\n@@ -181,7 +181,7 @@ static int indexed_table_ref_iter_next(void *p, struct reftable_record *rec)\n }\n \n int indexed_table_ref_iter_new(struct indexed_table_ref_iter **dest,\n-\t\t\t       struct reftable_reader *r, uint8_t *oid,\n+\t\t\t       struct reftable_table *t, uint8_t *oid,\n \t\t\t       int oid_len, uint64_t *offsets, int offset_len)\n {\n \tstruct indexed_table_ref_iter empty = INDEXED_TABLE_REF_ITER_INIT;\n@@ -195,7 +195,7 @@ int indexed_table_ref_iter_new(struct indexed_table_ref_iter **dest,\n \t}\n \n \t*itr = empty;\n-\titr->r = r;\n+\titr->table = t;\n \n \terr = reftable_buf_add(&itr->oid, oid, oid_len);\n \tif (err < 0)\ndiff --git a/reftable/iter.h b/reftable/iter.h\nindex bd217405f55..add5c9472fc 100644\n--- a/reftable/iter.h\n+++ b/reftable/iter.h\n@@ -59,7 +59,7 @@ void iterator_from_filtering_ref_iterator(struct reftable_iterator *,\n  * but using the object index.\n  */\n struct indexed_table_ref_iter {\n-\tstruct reftable_reader *r;\n+\tstruct reftable_table *table;\n \tstruct reftable_buf oid;\n \n \t/* mutable */\n@@ -83,7 +83,7 @@ void iterator_from_indexed_table_ref_iter(struct reftable_iterator *it,\n \n /* Takes ownership of `offsets` */\n int indexed_table_ref_iter_new(struct indexed_table_ref_iter **dest,\n-\t\t\t       struct reftable_reader *r, uint8_t *oid,\n+\t\t\t       struct reftable_table *t, uint8_t *oid,\n \t\t\t       int oid_len, uint64_t *offsets, int offset_len);\n \n #endif\ndiff --git a/reftable/merged.c b/reftable/merged.c\nindex 1829a081756..d5b974d660e 100644\n--- a/reftable/merged.c\n+++ b/reftable/merged.c\n@@ -11,11 +11,11 @@\n #include \"constants.h\"\n #include \"iter.h\"\n #include \"pq.h\"\n-#include \"reader.h\"\n #include \"record.h\"\n #include \"reftable-merged.h\"\n #include \"reftable-error.h\"\n #include \"system.h\"\n+#include \"table.h\"\n \n struct merged_subiter {\n \tstruct reftable_iterator iter;\n@@ -192,7 +192,7 @@ static void iterator_from_merged_iter(struct reftable_iterator *it,\n }\n \n int reftable_merged_table_new(struct reftable_merged_table **dest,\n-\t\t\t      struct reftable_reader **readers, size_t n,\n+\t\t\t      struct reftable_table **tables, size_t n,\n \t\t\t      enum reftable_hash hash_id)\n {\n \tstruct reftable_merged_table *m = NULL;\n@@ -200,10 +200,10 @@ int reftable_merged_table_new(struct reftable_merged_table **dest,\n \tuint64_t first_min = 0;\n \n \tfor (size_t i = 0; i < n; i++) {\n-\t\tuint64_t min = reftable_reader_min_update_index(readers[i]);\n-\t\tuint64_t max = reftable_reader_max_update_index(readers[i]);\n+\t\tuint64_t min = reftable_table_min_update_index(tables[i]);\n+\t\tuint64_t max = reftable_table_max_update_index(tables[i]);\n \n-\t\tif (reftable_reader_hash_id(readers[i]) != hash_id) {\n+\t\tif (reftable_table_hash_id(tables[i]) != hash_id) {\n \t\t\treturn REFTABLE_FORMAT_ERROR;\n \t\t}\n \t\tif (i == 0 || min < first_min) {\n@@ -218,8 +218,8 @@ int reftable_merged_table_new(struct reftable_merged_table **dest,\n \tif (!m)\n \t\treturn REFTABLE_OUT_OF_MEMORY_ERROR;\n \n-\tm->readers = readers;\n-\tm->readers_len = n;\n+\tm->tables = tables;\n+\tm->tables_len = n;\n \tm->min = first_min;\n \tm->max = last_max;\n \tm->hash_id = hash_id;\n@@ -254,20 +254,20 @@ int merged_table_init_iter(struct reftable_merged_table *mt,\n \tstruct merged_iter *mi = NULL;\n \tint ret;\n \n-\tif (mt->readers_len) {\n-\t\tREFTABLE_CALLOC_ARRAY(subiters, mt->readers_len);\n+\tif (mt->tables_len) {\n+\t\tREFTABLE_CALLOC_ARRAY(subiters, mt->tables_len);\n \t\tif (!subiters) {\n \t\t\tret = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\t\tgoto out;\n \t\t}\n \t}\n \n-\tfor (size_t i = 0; i < mt->readers_len; i++) {\n+\tfor (size_t i = 0; i < mt->tables_len; i++) {\n \t\tret = reftable_record_init(&subiters[i].rec, typ);\n \t\tif (ret < 0)\n \t\t\tgoto out;\n \n-\t\tret = reader_init_iter(mt->readers[i], &subiters[i].iter, typ);\n+\t\tret = table_init_iter(mt->tables[i], &subiters[i].iter, typ);\n \t\tif (ret < 0)\n \t\t\tgoto out;\n \t}\n@@ -280,14 +280,14 @@ int merged_table_init_iter(struct reftable_merged_table *mt,\n \tmi->advance_index = -1;\n \tmi->suppress_deletions = mt->suppress_deletions;\n \tmi->subiters = subiters;\n-\tmi->subiters_len = mt->readers_len;\n+\tmi->subiters_len = mt->tables_len;\n \n \titerator_from_merged_iter(it, mi);\n \tret = 0;\n \n out:\n \tif (ret < 0) {\n-\t\tfor (size_t i = 0; subiters && i < mt->readers_len; i++) {\n+\t\tfor (size_t i = 0; subiters && i < mt->tables_len; i++) {\n \t\t\treftable_iterator_destroy(&subiters[i].iter);\n \t\t\treftable_record_release(&subiters[i].rec);\n \t\t}\ndiff --git a/reftable/merged.h b/reftable/merged.h\nindex ae92409d342..4317e5f5f67 100644\n--- a/reftable/merged.h\n+++ b/reftable/merged.h\n@@ -13,8 +13,8 @@\n #include \"reftable-basics.h\"\n \n struct reftable_merged_table {\n-\tstruct reftable_reader **readers;\n-\tsize_t readers_len;\n+\tstruct reftable_table **tables;\n+\tsize_t tables_len;\n \tenum reftable_hash hash_id;\n \n \t/* If unset, produce deletions. This is useful for compaction. For the\ndiff --git a/reftable/reftable-merged.h b/reftable/reftable-merged.h\nindex 445756e475a..e5af846b32a 100644\n--- a/reftable/reftable-merged.h\n+++ b/reftable/reftable-merged.h\n@@ -26,14 +26,14 @@\n /* A merged table is implements seeking/iterating over a stack of tables. */\n struct reftable_merged_table;\n \n-struct reftable_reader;\n+struct reftable_table;\n \n /*\n- * reftable_merged_table_new creates a new merged table. The readers must be\n+ * reftable_merged_table_new creates a new merged table. The tables must be\n  * kept alive as long as the merged table is still in use.\n  */\n int reftable_merged_table_new(struct reftable_merged_table **dest,\n-\t\t\t      struct reftable_reader **readers, size_t n,\n+\t\t\t      struct reftable_table **tables, size_t n,\n \t\t\t      enum reftable_hash hash_id);\n \n /* Initialize a merged table iterator for reading refs. */\ndiff --git a/reftable/reftable-reader.h b/reftable/reftable-reader.h\ndeleted file mode 100644\nindex 9b6a8ac0716..00000000000\n--- a/reftable/reftable-reader.h\n+++ /dev/null\n@@ -1,72 +0,0 @@\n-/*\n- * Copyright 2020 Google LLC\n- *\n- * Use of this source code is governed by a BSD-style\n- * license that can be found in the LICENSE file or at\n- * https://developers.google.com/open-source/licenses/bsd\n- */\n-\n-#ifndef REFTABLE_READER_H\n-#define REFTABLE_READER_H\n-\n-#include \"reftable-iterator.h\"\n-#include \"reftable-blocksource.h\"\n-\n-/*\n- * Reading single tables\n- *\n- * The follow routines are for reading single files. For an\n- * application-level interface, skip ahead to struct\n- * reftable_merged_table and struct reftable_stack.\n- */\n-\n-/* The reader struct is a handle to an open reftable file. */\n-struct reftable_reader;\n-\n-/* reftable_reader_new opens a reftable for reading. If successful,\n- * returns 0 code and sets pp. The name is used for creating a\n- * stack. Typically, it is the basename of the file. The block source\n- * `src` is owned by the reader, and is closed on calling\n- * reftable_reader_destroy(). On error, the block source `src` is\n- * closed as well.\n- */\n-int reftable_reader_new(struct reftable_reader **pp,\n-\t\t\tstruct reftable_block_source *src, const char *name);\n-\n-/*\n- * Manage the reference count of the reftable reader. A newly initialized\n- * reader starts with a refcount of 1 and will be deleted once the refcount has\n- * reached 0.\n- *\n- * This is required because readers may have longer lifetimes than the stack\n- * they belong to. The stack may for example be reloaded while the old tables\n- * are still being accessed by an iterator.\n- */\n-void reftable_reader_incref(struct reftable_reader *reader);\n-void reftable_reader_decref(struct reftable_reader *reader);\n-\n-/* Initialize a reftable iterator for reading refs. */\n-int reftable_reader_init_ref_iterator(struct reftable_reader *r,\n-\t\t\t\t      struct reftable_iterator *it);\n-\n-/* Initialize a reftable iterator for reading logs. */\n-int reftable_reader_init_log_iterator(struct reftable_reader *r,\n-\t\t\t\t      struct reftable_iterator *it);\n-\n-/* returns the hash ID used in this table. */\n-enum reftable_hash reftable_reader_hash_id(struct reftable_reader *r);\n-\n-/* return an iterator for the refs pointing to `oid`. */\n-int reftable_reader_refs_for(struct reftable_reader *r,\n-\t\t\t     struct reftable_iterator *it, uint8_t *oid);\n-\n-/* return the max_update_index for a table */\n-uint64_t reftable_reader_max_update_index(struct reftable_reader *r);\n-\n-/* return the min_update_index for a table */\n-uint64_t reftable_reader_min_update_index(struct reftable_reader *r);\n-\n-/* print blocks onto stdout for debugging. */\n-int reftable_reader_print_blocks(const char *tablename);\n-\n-#endif\ndiff --git a/reftable/reftable-table.h b/reftable/reftable-table.h\nnew file mode 100644\nindex 00000000000..94379026727\n--- /dev/null\n+++ b/reftable/reftable-table.h\n@@ -0,0 +1,72 @@\n+/*\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n+\n+#ifndef REFTABLE_TABLE_H\n+#define REFTABLE_TABLE_H\n+\n+#include \"reftable-iterator.h\"\n+#include \"reftable-blocksource.h\"\n+\n+/*\n+ * Reading single tables\n+ *\n+ * The follow routines are for reading single files. For an\n+ * application-level interface, skip ahead to struct\n+ * reftable_merged_table and struct reftable_stack.\n+ */\n+\n+/* The table struct is a handle to an open reftable file. */\n+struct reftable_table;\n+\n+/* reftable_table_new opens a reftable for reading. If successful,\n+ * returns 0 code and sets pp. The name is used for creating a\n+ * stack. Typically, it is the basename of the file. The block source\n+ * `src` is owned by the table, and is closed on calling\n+ * reftable_table_destroy(). On error, the block source `src` is\n+ * closed as well.\n+ */\n+int reftable_table_new(struct reftable_table **out,\n+\t\t       struct reftable_block_source *src, const char *name);\n+\n+/*\n+ * Manage the reference count of the reftable table. A newly initialized\n+ * table starts with a refcount of 1 and will be deleted once the refcount has\n+ * reached 0.\n+ *\n+ * This is required because tables may have longer lifetimes than the stack\n+ * they belong to. The stack may for example be reloaded while the old tables\n+ * are still being accessed by an iterator.\n+ */\n+void reftable_table_incref(struct reftable_table *table);\n+void reftable_table_decref(struct reftable_table *table);\n+\n+/* Initialize a reftable iterator for reading refs. */\n+int reftable_table_init_ref_iterator(struct reftable_table *t,\n+\t\t\t\t     struct reftable_iterator *it);\n+\n+/* Initialize a reftable iterator for reading logs. */\n+int reftable_table_init_log_iterator(struct reftable_table *t,\n+\t\t\t\t     struct reftable_iterator *it);\n+\n+/* returns the hash ID used in this table. */\n+enum reftable_hash reftable_table_hash_id(struct reftable_table *t);\n+\n+/* return an iterator for the refs pointing to `oid`. */\n+int reftable_table_refs_for(struct reftable_table *t,\n+\t\t\t    struct reftable_iterator *it, uint8_t *oid);\n+\n+/* return the max_update_index for a table */\n+uint64_t reftable_table_max_update_index(struct reftable_table *t);\n+\n+/* return the min_update_index for a table */\n+uint64_t reftable_table_min_update_index(struct reftable_table *t);\n+\n+/* print blocks onto stdout for debugging. */\n+int reftable_table_print_blocks(const char *tablename);\n+\n+#endif\ndiff --git a/reftable/stack.c b/reftable/stack.c\nindex bc3cfa91170..cc48e725b14 100644\n--- a/reftable/stack.c\n+++ b/reftable/stack.c\n@@ -11,10 +11,10 @@\n #include \"system.h\"\n #include \"constants.h\"\n #include \"merged.h\"\n-#include \"reader.h\"\n #include \"reftable-error.h\"\n #include \"reftable-record.h\"\n #include \"reftable-merged.h\"\n+#include \"table.h\"\n #include \"writer.h\"\n \n static int stack_try_add(struct reftable_stack *st,\n@@ -248,11 +248,11 @@ void reftable_stack_destroy(struct reftable_stack *st)\n \t\tREFTABLE_FREE_AND_NULL(names);\n \t}\n \n-\tif (st->readers) {\n+\tif (st->tables) {\n \t\tstruct reftable_buf filename = REFTABLE_BUF_INIT;\n \n-\t\tfor (size_t i = 0; i < st->readers_len; i++) {\n-\t\t\tconst char *name = reader_name(st->readers[i]);\n+\t\tfor (size_t i = 0; i < st->tables_len; i++) {\n+\t\t\tconst char *name = reftable_table_name(st->tables[i]);\n \t\t\tint try_unlinking = 1;\n \n \t\t\treftable_buf_reset(&filename);\n@@ -260,7 +260,7 @@ void reftable_stack_destroy(struct reftable_stack *st)\n \t\t\t\tif (stack_filename(&filename, st, name) < 0)\n \t\t\t\t\ttry_unlinking = 0;\n \t\t\t}\n-\t\t\treftable_reader_decref(st->readers[i]);\n+\t\t\treftable_table_decref(st->tables[i]);\n \n \t\t\tif (try_unlinking && filename.len) {\n \t\t\t\t/* On Windows, can only unlink after closing. */\n@@ -269,8 +269,8 @@ void reftable_stack_destroy(struct reftable_stack *st)\n \t\t}\n \n \t\treftable_buf_release(&filename);\n-\t\tst->readers_len = 0;\n-\t\tREFTABLE_FREE_AND_NULL(st->readers);\n+\t\tst->tables_len = 0;\n+\t\tREFTABLE_FREE_AND_NULL(st->tables);\n \t}\n \n \tif (st->list_fd >= 0) {\n@@ -284,14 +284,14 @@ void reftable_stack_destroy(struct reftable_stack *st)\n \tfree_names(names);\n }\n \n-static struct reftable_reader **stack_copy_readers(struct reftable_stack *st,\n-\t\t\t\t\t\t   size_t cur_len)\n+static struct reftable_table **stack_copy_tables(struct reftable_stack *st,\n+\t\t\t\t\t\t size_t cur_len)\n {\n-\tstruct reftable_reader **cur = reftable_calloc(cur_len, sizeof(*cur));\n+\tstruct reftable_table **cur = reftable_calloc(cur_len, sizeof(*cur));\n \tif (!cur)\n \t\treturn NULL;\n \tfor (size_t i = 0; i < cur_len; i++)\n-\t\tcur[i] = st->readers[i];\n+\t\tcur[i] = st->tables[i];\n \treturn cur;\n }\n \n@@ -299,19 +299,19 @@ static int reftable_stack_reload_once(struct reftable_stack *st,\n \t\t\t\t      const char **names,\n \t\t\t\t      int reuse_open)\n {\n-\tsize_t cur_len = !st->merged ? 0 : st->merged->readers_len;\n-\tstruct reftable_reader **cur = NULL;\n-\tstruct reftable_reader **reused = NULL;\n-\tstruct reftable_reader **new_readers = NULL;\n+\tsize_t cur_len = !st->merged ? 0 : st->merged->tables_len;\n+\tstruct reftable_table **cur = NULL;\n+\tstruct reftable_table **reused = NULL;\n+\tstruct reftable_table **new_tables = NULL;\n \tsize_t reused_len = 0, reused_alloc = 0, names_len;\n-\tsize_t new_readers_len = 0;\n+\tsize_t new_tables_len = 0;\n \tstruct reftable_merged_table *new_merged = NULL;\n \tstruct reftable_buf table_path = REFTABLE_BUF_INIT;\n \tint err = 0;\n \tsize_t i;\n \n \tif (cur_len) {\n-\t\tcur = stack_copy_readers(st, cur_len);\n+\t\tcur = stack_copy_tables(st, cur_len);\n \t\tif (!cur) {\n \t\t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\t\tgoto done;\n@@ -321,28 +321,28 @@ static int reftable_stack_reload_once(struct reftable_stack *st,\n \tnames_len = names_length(names);\n \n \tif (names_len) {\n-\t\tnew_readers = reftable_calloc(names_len, sizeof(*new_readers));\n-\t\tif (!new_readers) {\n+\t\tnew_tables = reftable_calloc(names_len, sizeof(*new_tables));\n+\t\tif (!new_tables) {\n \t\t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\t\tgoto done;\n \t\t}\n \t}\n \n \twhile (*names) {\n-\t\tstruct reftable_reader *rd = NULL;\n+\t\tstruct reftable_table *table = NULL;\n \t\tconst char *name = *names++;\n \n \t\t/* this is linear; we assume compaction keeps the number of\n \t\t   tables under control so this is not quadratic. */\n \t\tfor (i = 0; reuse_open && i < cur_len; i++) {\n \t\t\tif (cur[i] && 0 == strcmp(cur[i]->name, name)) {\n-\t\t\t\trd = cur[i];\n+\t\t\t\ttable = cur[i];\n \t\t\t\tcur[i] = NULL;\n \n \t\t\t\t/*\n \t\t\t\t * When reloading the stack fails, we end up\n-\t\t\t\t * releasing all new readers. This also\n-\t\t\t\t * includes the reused readers, even though\n+\t\t\t\t * releasing all new tables. This also\n+\t\t\t\t * includes the reused tables, even though\n \t\t\t\t * they are still in used by the old stack. We\n \t\t\t\t * thus need to keep them alive here, which we\n \t\t\t\t * do by bumping their refcount.\n@@ -354,13 +354,13 @@ static int reftable_stack_reload_once(struct reftable_stack *st,\n \t\t\t\t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\t\t\t\tgoto done;\n \t\t\t\t}\n-\t\t\t\treused[reused_len++] = rd;\n-\t\t\t\treftable_reader_incref(rd);\n+\t\t\t\treused[reused_len++] = table;\n+\t\t\t\treftable_table_incref(table);\n \t\t\t\tbreak;\n \t\t\t}\n \t\t}\n \n-\t\tif (!rd) {\n+\t\tif (!table) {\n \t\t\tstruct reftable_block_source src = { NULL };\n \n \t\t\terr = stack_filename(&table_path, st, name);\n@@ -372,36 +372,36 @@ static int reftable_stack_reload_once(struct reftable_stack *st,\n \t\t\tif (err < 0)\n \t\t\t\tgoto done;\n \n-\t\t\terr = reftable_reader_new(&rd, &src, name);\n+\t\t\terr = reftable_table_new(&table, &src, name);\n \t\t\tif (err < 0)\n \t\t\t\tgoto done;\n \t\t}\n \n-\t\tnew_readers[new_readers_len] = rd;\n-\t\tnew_readers_len++;\n+\t\tnew_tables[new_tables_len] = table;\n+\t\tnew_tables_len++;\n \t}\n \n \t/* success! */\n-\terr = reftable_merged_table_new(&new_merged, new_readers,\n-\t\t\t\t\tnew_readers_len, st->opts.hash_id);\n+\terr = reftable_merged_table_new(&new_merged, new_tables,\n+\t\t\t\t\tnew_tables_len, st->opts.hash_id);\n \tif (err < 0)\n \t\tgoto done;\n \n \t/*\n-\t * Close the old, non-reused readers and proactively try to unlink\n+\t * Close the old, non-reused tables and proactively try to unlink\n \t * them. This is done for systems like Windows, where the underlying\n-\t * file of such an open reader wouldn't have been possible to be\n+\t * file of such an open table wouldn't have been possible to be\n \t * unlinked by the compacting process.\n \t */\n \tfor (i = 0; i < cur_len; i++) {\n \t\tif (cur[i]) {\n-\t\t\tconst char *name = reader_name(cur[i]);\n+\t\t\tconst char *name = reftable_table_name(cur[i]);\n \n \t\t\terr = stack_filename(&table_path, st, name);\n \t\t\tif (err < 0)\n \t\t\t\tgoto done;\n \n-\t\t\treftable_reader_decref(cur[i]);\n+\t\t\treftable_table_decref(cur[i]);\n \t\t\tunlink(table_path.buf);\n \t\t}\n \t}\n@@ -412,25 +412,25 @@ static int reftable_stack_reload_once(struct reftable_stack *st,\n \tnew_merged->suppress_deletions = 1;\n \tst->merged = new_merged;\n \n-\tif (st->readers)\n-\t\treftable_free(st->readers);\n-\tst->readers = new_readers;\n-\tst->readers_len = new_readers_len;\n-\tnew_readers = NULL;\n-\tnew_readers_len = 0;\n+\tif (st->tables)\n+\t\treftable_free(st->tables);\n+\tst->tables = new_tables;\n+\tst->tables_len = new_tables_len;\n+\tnew_tables = NULL;\n+\tnew_tables_len = 0;\n \n \t/*\n-\t * Decrement the refcount of reused readers again. This only needs to\n+\t * Decrement the refcount of reused tables again. This only needs to\n \t * happen on the successful case, because on the unsuccessful one we\n-\t * decrement their refcount via `new_readers`.\n+\t * decrement their refcount via `new_tables`.\n \t */\n \tfor (i = 0; i < reused_len; i++)\n-\t\treftable_reader_decref(reused[i]);\n+\t\treftable_table_decref(reused[i]);\n \n done:\n-\tfor (i = 0; i < new_readers_len; i++)\n-\t\treftable_reader_decref(new_readers[i]);\n-\treftable_free(new_readers);\n+\tfor (i = 0; i < new_tables_len; i++)\n+\t\treftable_table_decref(new_tables[i]);\n+\treftable_free(new_tables);\n \treftable_free(reused);\n \treftable_free(cur);\n \treftable_buf_release(&table_path);\n@@ -615,10 +615,10 @@ static int stack_uptodate(struct reftable_stack *st)\n \t\t\t/*\n \t\t\t * It's fine for \"tables.list\" to not exist. In that\n \t\t\t * case, we have to refresh when the loaded stack has\n-\t\t\t * any readers.\n+\t\t\t * any tables.\n \t\t\t */\n \t\t\tif (errno == ENOENT)\n-\t\t\t\treturn !!st->readers_len;\n+\t\t\t\treturn !!st->tables_len;\n \t\t\treturn REFTABLE_IO_ERROR;\n \t\t}\n \n@@ -637,19 +637,19 @@ static int stack_uptodate(struct reftable_stack *st)\n \tif (err < 0)\n \t\treturn err;\n \n-\tfor (size_t i = 0; i < st->readers_len; i++) {\n+\tfor (size_t i = 0; i < st->tables_len; i++) {\n \t\tif (!names[i]) {\n \t\t\terr = 1;\n \t\t\tgoto done;\n \t\t}\n \n-\t\tif (strcmp(st->readers[i]->name, names[i])) {\n+\t\tif (strcmp(st->tables[i]->name, names[i])) {\n \t\t\terr = 1;\n \t\t\tgoto done;\n \t\t}\n \t}\n \n-\tif (names[st->merged->readers_len]) {\n+\tif (names[st->merged->tables_len]) {\n \t\terr = 1;\n \t\tgoto done;\n \t}\n@@ -792,8 +792,8 @@ int reftable_addition_commit(struct reftable_addition *add)\n \tif (add->new_tables_len == 0)\n \t\tgoto done;\n \n-\tfor (i = 0; i < add->stack->merged->readers_len; i++) {\n-\t\tif ((err = reftable_buf_addstr(&table_list, add->stack->readers[i]->name)) < 0 ||\n+\tfor (i = 0; i < add->stack->merged->tables_len; i++) {\n+\t\tif ((err = reftable_buf_addstr(&table_list, add->stack->tables[i]->name)) < 0 ||\n \t\t    (err = reftable_buf_addstr(&table_list, \"\\n\")) < 0)\n \t\t\tgoto done;\n \t}\n@@ -1000,9 +1000,9 @@ int reftable_addition_add(struct reftable_addition *add,\n \n uint64_t reftable_stack_next_update_index(struct reftable_stack *st)\n {\n-\tint sz = st->merged->readers_len;\n+\tint sz = st->merged->tables_len;\n \tif (sz > 0)\n-\t\treturn reftable_reader_max_update_index(st->readers[sz - 1]) +\n+\t\treturn reftable_table_max_update_index(st->tables[sz - 1]) +\n \t\t       1;\n \treturn 1;\n }\n@@ -1021,8 +1021,8 @@ static int stack_compact_locked(struct reftable_stack *st,\n \tstruct reftable_tmpfile tab_file = REFTABLE_TMPFILE_INIT;\n \tint err = 0;\n \n-\terr = format_name(&next_name, reftable_reader_min_update_index(st->readers[first]),\n-\t\t\t  reftable_reader_max_update_index(st->readers[last]));\n+\terr = format_name(&next_name, reftable_table_min_update_index(st->tables[first]),\n+\t\t\t  reftable_table_max_update_index(st->tables[last]));\n \tif (err < 0)\n \t\tgoto done;\n \n@@ -1087,13 +1087,13 @@ static int stack_write_compact(struct reftable_stack *st,\n \tint err = 0;\n \n \tfor (size_t i = first; i <= last; i++)\n-\t\tst->stats.bytes += st->readers[i]->size;\n-\terr = reftable_writer_set_limits(wr, st->readers[first]->min_update_index,\n-\t\t\t\t\t st->readers[last]->max_update_index);\n+\t\tst->stats.bytes += st->tables[i]->size;\n+\terr = reftable_writer_set_limits(wr, st->tables[first]->min_update_index,\n+\t\t\t\t\t st->tables[last]->max_update_index);\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = reftable_merged_table_new(&mt, st->readers + first, subtabs_len,\n+\terr = reftable_merged_table_new(&mt, st->tables + first, subtabs_len,\n \t\t\t\t\tst->opts.hash_id);\n \tif (err < 0)\n \t\tgoto done;\n@@ -1250,7 +1250,7 @@ static int stack_compact_range(struct reftable_stack *st,\n \t\ttable_locks[i] = REFTABLE_FLOCK_INIT;\n \n \tfor (i = last + 1; i > first; i--) {\n-\t\terr = stack_filename(&table_name, st, reader_name(st->readers[i - 1]));\n+\t\terr = stack_filename(&table_name, st, reftable_table_name(st->tables[i - 1]));\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n@@ -1376,7 +1376,7 @@ static int stack_compact_range(struct reftable_stack *st,\n \t\t * compacted in the updated \"tables.list\" file.\n \t\t */\n \t\tfor (size_t i = 0; names[i]; i++) {\n-\t\t\tif (strcmp(names[i], st->readers[first]->name))\n+\t\t\tif (strcmp(names[i], st->tables[first]->name))\n \t\t\t\tcontinue;\n \n \t\t\t/*\n@@ -1386,8 +1386,8 @@ static int stack_compact_range(struct reftable_stack *st,\n \t\t\t * have compacted them.\n \t\t\t */\n \t\t\tfor (size_t j = 1; j < last - first + 1; j++) {\n-\t\t\t\tconst char *old = first + j < st->merged->readers_len ?\n-\t\t\t\t\tst->readers[first + j]->name : NULL;\n+\t\t\t\tconst char *old = first + j < st->merged->tables_len ?\n+\t\t\t\t\tst->tables[first + j]->name : NULL;\n \t\t\t\tconst char *new = names[i + j];\n \n \t\t\t\t/*\n@@ -1427,16 +1427,16 @@ static int stack_compact_range(struct reftable_stack *st,\n \t\t * `fd_read_lines()` uses a `NULL` sentinel to indicate that\n \t\t * the array is at its end. As we use `free_names()` to free\n \t\t * the array, we need to include this sentinel value here and\n-\t\t * thus have to allocate `readers_len + 1` many entries.\n+\t\t * thus have to allocate `tables_len + 1` many entries.\n \t\t */\n-\t\tREFTABLE_CALLOC_ARRAY(names, st->merged->readers_len + 1);\n+\t\tREFTABLE_CALLOC_ARRAY(names, st->merged->tables_len + 1);\n \t\tif (!names) {\n \t\t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\t\tgoto done;\n \t\t}\n \n-\t\tfor (size_t i = 0; i < st->merged->readers_len; i++) {\n-\t\t\tnames[i] = reftable_strdup(st->readers[i]->name);\n+\t\tfor (size_t i = 0; i < st->merged->tables_len; i++) {\n+\t\t\tnames[i] = reftable_strdup(st->tables[i]->name);\n \t\t\tif (!names[i]) {\n \t\t\t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\t\t\tgoto done;\n@@ -1451,8 +1451,8 @@ static int stack_compact_range(struct reftable_stack *st,\n \t * it into place now.\n \t */\n \tif (!is_empty_table) {\n-\t\terr = format_name(&new_table_name, st->readers[first]->min_update_index,\n-\t\t\t\t  st->readers[last]->max_update_index);\n+\t\terr = format_name(&new_table_name, st->tables[first]->min_update_index,\n+\t\t\t\t  st->tables[last]->max_update_index);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n@@ -1559,7 +1559,7 @@ static int stack_compact_range(struct reftable_stack *st,\n int reftable_stack_compact_all(struct reftable_stack *st,\n \t\t\t       struct reftable_log_expiry_config *config)\n {\n-\tsize_t last = st->merged->readers_len ? st->merged->readers_len - 1 : 0;\n+\tsize_t last = st->merged->tables_len ? st->merged->tables_len - 1 : 0;\n \treturn stack_compact_range(st, 0, last, config, 0);\n }\n \n@@ -1650,12 +1650,12 @@ static uint64_t *stack_table_sizes_for_compaction(struct reftable_stack *st)\n \tint overhead = header_size(version) - 1;\n \tuint64_t *sizes;\n \n-\tREFTABLE_CALLOC_ARRAY(sizes, st->merged->readers_len);\n+\tREFTABLE_CALLOC_ARRAY(sizes, st->merged->tables_len);\n \tif (!sizes)\n \t\treturn NULL;\n \n-\tfor (size_t i = 0; i < st->merged->readers_len; i++)\n-\t\tsizes[i] = st->readers[i]->size - overhead;\n+\tfor (size_t i = 0; i < st->merged->tables_len; i++)\n+\t\tsizes[i] = st->tables[i]->size - overhead;\n \n \treturn sizes;\n }\n@@ -1665,14 +1665,14 @@ int reftable_stack_auto_compact(struct reftable_stack *st)\n \tstruct segment seg;\n \tuint64_t *sizes;\n \n-\tif (st->merged->readers_len < 2)\n+\tif (st->merged->tables_len < 2)\n \t\treturn 0;\n \n \tsizes = stack_table_sizes_for_compaction(st);\n \tif (!sizes)\n \t\treturn REFTABLE_OUT_OF_MEMORY_ERROR;\n \n-\tseg = suggest_compaction_segment(sizes, st->merged->readers_len,\n+\tseg = suggest_compaction_segment(sizes, st->merged->tables_len,\n \t\t\t\t\t st->opts.auto_compaction_factor);\n \treftable_free(sizes);\n \n@@ -1763,7 +1763,7 @@ static void remove_maybe_stale_table(struct reftable_stack *st, uint64_t max,\n \tint err = 0;\n \tuint64_t update_idx = 0;\n \tstruct reftable_block_source src = { NULL };\n-\tstruct reftable_reader *rd = NULL;\n+\tstruct reftable_table *table = NULL;\n \tstruct reftable_buf table_path = REFTABLE_BUF_INIT;\n \n \terr = stack_filename(&table_path, st, name);\n@@ -1774,12 +1774,12 @@ static void remove_maybe_stale_table(struct reftable_stack *st, uint64_t max,\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = reftable_reader_new(&rd, &src, name);\n+\terr = reftable_table_new(&table, &src, name);\n \tif (err < 0)\n \t\tgoto done;\n \n-\tupdate_idx = reftable_reader_max_update_index(rd);\n-\treftable_reader_decref(rd);\n+\tupdate_idx = reftable_table_max_update_index(table);\n+\treftable_table_decref(table);\n \n \tif (update_idx <= max) {\n \t\tunlink(table_path.buf);\n@@ -1803,8 +1803,8 @@ static int reftable_stack_clean_locked(struct reftable_stack *st)\n \t\tif (!is_table_name(d->d_name))\n \t\t\tcontinue;\n \n-\t\tfor (size_t i = 0; !found && i < st->readers_len; i++)\n-\t\t\tfound = !strcmp(reader_name(st->readers[i]), d->d_name);\n+\t\tfor (size_t i = 0; !found && i < st->tables_len; i++)\n+\t\t\tfound = !strcmp(reftable_table_name(st->tables[i]), d->d_name);\n \t\tif (found)\n \t\t\tcontinue;\n \ndiff --git a/reftable/stack.h b/reftable/stack.h\nindex d91f5ce6072..bc28f2998ac 100644\n--- a/reftable/stack.h\n+++ b/reftable/stack.h\n@@ -22,8 +22,8 @@ struct reftable_stack {\n \n \tstruct reftable_write_options opts;\n \n-\tstruct reftable_reader **readers;\n-\tsize_t readers_len;\n+\tstruct reftable_table **tables;\n+\tsize_t tables_len;\n \tstruct reftable_merged_table *merged;\n \tstruct reftable_compaction_stats stats;\n };\ndiff --git a/reftable/reader.c b/reftable/table.c\nsimilarity index 73%\nrename from reftable/reader.c\nrename to reftable/table.c\nindex 155a26c30ff..440fb559ad1 100644\n--- a/reftable/reader.c\n+++ b/reftable/table.c\n@@ -6,7 +6,7 @@\n  * https://developers.google.com/open-source/licenses/bsd\n  */\n \n-#include \"reader.h\"\n+#include \"table.h\"\n \n #include \"system.h\"\n #include \"block.h\"\n@@ -39,48 +39,48 @@ void block_source_close(struct reftable_block_source *source)\n \tsource->ops = NULL;\n }\n \n-static struct reftable_reader_offsets *\n-reader_offsets_for(struct reftable_reader *r, uint8_t typ)\n+static struct reftable_table_offsets *\n+table_offsets_for(struct reftable_table *t, uint8_t typ)\n {\n \tswitch (typ) {\n \tcase BLOCK_TYPE_REF:\n-\t\treturn &r->ref_offsets;\n+\t\treturn &t->ref_offsets;\n \tcase BLOCK_TYPE_LOG:\n-\t\treturn &r->log_offsets;\n+\t\treturn &t->log_offsets;\n \tcase BLOCK_TYPE_OBJ:\n-\t\treturn &r->obj_offsets;\n+\t\treturn &t->obj_offsets;\n \t}\n \tabort();\n }\n \n-static int reader_get_block(struct reftable_reader *r,\n-\t\t\t    struct reftable_block *dest, uint64_t off,\n-\t\t\t    uint32_t sz)\n+static int table_get_block(struct reftable_table *t,\n+\t\t\t   struct reftable_block *dest, uint64_t off,\n+\t\t\t   uint32_t sz)\n {\n \tssize_t bytes_read;\n-\tif (off >= r->size)\n+\tif (off >= t->size)\n \t\treturn 0;\n-\tif (off + sz > r->size)\n-\t\tsz = r->size - off;\n+\tif (off + sz > t->size)\n+\t\tsz = t->size - off;\n \n-\tbytes_read = block_source_read_block(&r->source, dest, off, sz);\n+\tbytes_read = block_source_read_block(&t->source, dest, off, sz);\n \tif (bytes_read < 0)\n \t\treturn (int)bytes_read;\n \n \treturn 0;\n }\n \n-enum reftable_hash reftable_reader_hash_id(struct reftable_reader *r)\n+enum reftable_hash reftable_table_hash_id(struct reftable_table *t)\n {\n-\treturn r->hash_id;\n+\treturn t->hash_id;\n }\n \n-const char *reader_name(struct reftable_reader *r)\n+const char *reftable_table_name(struct reftable_table *t)\n {\n-\treturn r->name;\n+\treturn t->name;\n }\n \n-static int parse_footer(struct reftable_reader *r, uint8_t *footer,\n+static int parse_footer(struct reftable_table *t, uint8_t *footer,\n \t\t\tuint8_t *header)\n {\n \tuint8_t *f = footer;\n@@ -95,29 +95,29 @@ static int parse_footer(struct reftable_reader *r, uint8_t *footer,\n \t}\n \tf += 4;\n \n-\tif (memcmp(footer, header, header_size(r->version))) {\n+\tif (memcmp(footer, header, header_size(t->version))) {\n \t\terr = REFTABLE_FORMAT_ERROR;\n \t\tgoto done;\n \t}\n \n \tf++;\n-\tr->block_size = reftable_get_be24(f);\n+\tt->block_size = reftable_get_be24(f);\n \n \tf += 3;\n-\tr->min_update_index = reftable_get_be64(f);\n+\tt->min_update_index = reftable_get_be64(f);\n \tf += 8;\n-\tr->max_update_index = reftable_get_be64(f);\n+\tt->max_update_index = reftable_get_be64(f);\n \tf += 8;\n \n-\tif (r->version == 1) {\n-\t\tr->hash_id = REFTABLE_HASH_SHA1;\n+\tif (t->version == 1) {\n+\t\tt->hash_id = REFTABLE_HASH_SHA1;\n \t} else {\n \t\tswitch (reftable_get_be32(f)) {\n \t\tcase REFTABLE_FORMAT_ID_SHA1:\n-\t\t\tr->hash_id = REFTABLE_HASH_SHA1;\n+\t\t\tt->hash_id = REFTABLE_HASH_SHA1;\n \t\t\tbreak;\n \t\tcase REFTABLE_FORMAT_ID_SHA256:\n-\t\t\tr->hash_id = REFTABLE_HASH_SHA256;\n+\t\t\tt->hash_id = REFTABLE_HASH_SHA256;\n \t\t\tbreak;\n \t\tdefault:\n \t\t\terr = REFTABLE_FORMAT_ERROR;\n@@ -127,20 +127,20 @@ static int parse_footer(struct reftable_reader *r, uint8_t *footer,\n \t\tf += 4;\n \t}\n \n-\tr->ref_offsets.index_offset = reftable_get_be64(f);\n+\tt->ref_offsets.index_offset = reftable_get_be64(f);\n \tf += 8;\n \n-\tr->obj_offsets.offset = reftable_get_be64(f);\n+\tt->obj_offsets.offset = reftable_get_be64(f);\n \tf += 8;\n \n-\tr->object_id_len = r->obj_offsets.offset & ((1 << 5) - 1);\n-\tr->obj_offsets.offset >>= 5;\n+\tt->object_id_len = t->obj_offsets.offset & ((1 << 5) - 1);\n+\tt->obj_offsets.offset >>= 5;\n \n-\tr->obj_offsets.index_offset = reftable_get_be64(f);\n+\tt->obj_offsets.index_offset = reftable_get_be64(f);\n \tf += 8;\n-\tr->log_offsets.offset = reftable_get_be64(f);\n+\tt->log_offsets.offset = reftable_get_be64(f);\n \tf += 8;\n-\tr->log_offsets.index_offset = reftable_get_be64(f);\n+\tt->log_offsets.index_offset = reftable_get_be64(f);\n \tf += 8;\n \n \tcomputed_crc = crc32(0, footer, f - footer);\n@@ -151,13 +151,13 @@ static int parse_footer(struct reftable_reader *r, uint8_t *footer,\n \t\tgoto done;\n \t}\n \n-\tfirst_block_typ = header[header_size(r->version)];\n-\tr->ref_offsets.is_present = (first_block_typ == BLOCK_TYPE_REF);\n-\tr->ref_offsets.offset = 0;\n-\tr->log_offsets.is_present = (first_block_typ == BLOCK_TYPE_LOG ||\n-\t\t\t\t     r->log_offsets.offset > 0);\n-\tr->obj_offsets.is_present = r->obj_offsets.offset > 0;\n-\tif (r->obj_offsets.is_present && !r->object_id_len) {\n+\tfirst_block_typ = header[header_size(t->version)];\n+\tt->ref_offsets.is_present = (first_block_typ == BLOCK_TYPE_REF);\n+\tt->ref_offsets.offset = 0;\n+\tt->log_offsets.is_present = (first_block_typ == BLOCK_TYPE_LOG ||\n+\t\t\t\t     t->log_offsets.offset > 0);\n+\tt->obj_offsets.is_present = t->obj_offsets.offset > 0;\n+\tif (t->obj_offsets.is_present && !t->object_id_len) {\n \t\terr = REFTABLE_FORMAT_ERROR;\n \t\tgoto done;\n \t}\n@@ -168,7 +168,7 @@ static int parse_footer(struct reftable_reader *r, uint8_t *footer,\n }\n \n struct table_iter {\n-\tstruct reftable_reader *r;\n+\tstruct reftable_table *table;\n \tuint8_t typ;\n \tuint64_t block_off;\n \tstruct block_reader br;\n@@ -176,12 +176,12 @@ struct table_iter {\n \tint is_finished;\n };\n \n-static int table_iter_init(struct table_iter *ti, struct reftable_reader *r)\n+static int table_iter_init(struct table_iter *ti, struct reftable_table *t)\n {\n \tstruct block_iter bi = BLOCK_ITER_INIT;\n \tmemset(ti, 0, sizeof(*ti));\n-\treftable_reader_incref(r);\n-\tti->r = r;\n+\treftable_table_incref(t);\n+\tti->table = t;\n \tti->bi = bi;\n \treturn 0;\n }\n@@ -191,7 +191,7 @@ static int table_iter_next_in_block(struct table_iter *ti,\n {\n \tint res = block_iter_next(&ti->bi, rec);\n \tif (res == 0 && reftable_record_type(rec) == BLOCK_TYPE_REF) {\n-\t\trec->u.ref.update_index += ti->r->min_update_index;\n+\t\trec->u.ref.update_index += ti->table->min_update_index;\n \t}\n \n \treturn res;\n@@ -219,26 +219,26 @@ static int32_t extract_block_size(uint8_t *data, uint8_t *typ, uint64_t off,\n \treturn result;\n }\n \n-int reader_init_block_reader(struct reftable_reader *r, struct block_reader *br,\n-\t\t\t     uint64_t next_off, uint8_t want_typ)\n+int table_init_block_reader(struct reftable_table *t, struct block_reader *br,\n+\t\t\t    uint64_t next_off, uint8_t want_typ)\n {\n-\tint32_t guess_block_size = r->block_size ? r->block_size :\n+\tint32_t guess_block_size = t->block_size ? t->block_size :\n \t\t\t\t\t\t\t DEFAULT_BLOCK_SIZE;\n \tstruct reftable_block block = { NULL };\n \tuint8_t block_typ = 0;\n \tint err = 0;\n-\tuint32_t header_off = next_off ? 0 : header_size(r->version);\n+\tuint32_t header_off = next_off ? 0 : header_size(t->version);\n \tint32_t block_size = 0;\n \n-\tif (next_off >= r->size)\n+\tif (next_off >= t->size)\n \t\treturn 1;\n \n-\terr = reader_get_block(r, &block, next_off, guess_block_size);\n+\terr = table_get_block(t, &block, next_off, guess_block_size);\n \tif (err < 0)\n \t\tgoto done;\n \n \tblock_size = extract_block_size(block.data, &block_typ, next_off,\n-\t\t\t\t\tr->version);\n+\t\t\t\t\tt->version);\n \tif (block_size < 0) {\n \t\terr = block_size;\n \t\tgoto done;\n@@ -250,14 +250,14 @@ int reader_init_block_reader(struct reftable_reader *r, struct block_reader *br,\n \n \tif (block_size > guess_block_size) {\n \t\treftable_block_done(&block);\n-\t\terr = reader_get_block(r, &block, next_off, block_size);\n+\t\terr = table_get_block(t, &block, next_off, block_size);\n \t\tif (err < 0) {\n \t\t\tgoto done;\n \t\t}\n \t}\n \n-\terr = block_reader_init(br, &block, header_off, r->block_size,\n-\t\t\t\thash_size(r->hash_id));\n+\terr = block_reader_init(br, &block, header_off, t->block_size,\n+\t\t\t\thash_size(t->hash_id));\n done:\n \treftable_block_done(&block);\n \n@@ -268,7 +268,7 @@ static void table_iter_close(struct table_iter *ti)\n {\n \ttable_iter_block_done(ti);\n \tblock_iter_close(&ti->bi);\n-\treftable_reader_decref(ti->r);\n+\treftable_table_decref(ti->table);\n }\n \n static int table_iter_next_block(struct table_iter *ti)\n@@ -276,7 +276,7 @@ static int table_iter_next_block(struct table_iter *ti)\n \tuint64_t next_block_off = ti->block_off + ti->br.full_block_size;\n \tint err;\n \n-\terr = reader_init_block_reader(ti->r, &ti->br, next_block_off, ti->typ);\n+\terr = table_init_block_reader(ti->table, &ti->br, next_block_off, ti->typ);\n \tif (err > 0)\n \t\tti->is_finished = 1;\n \tif (err)\n@@ -326,7 +326,7 @@ static int table_iter_seek_to(struct table_iter *ti, uint64_t off, uint8_t typ)\n {\n \tint err;\n \n-\terr = reader_init_block_reader(ti->r, &ti->br, off, typ);\n+\terr = table_init_block_reader(ti->table, &ti->br, off, typ);\n \tif (err != 0)\n \t\treturn err;\n \n@@ -339,7 +339,7 @@ static int table_iter_seek_to(struct table_iter *ti, uint64_t off, uint8_t typ)\n \n static int table_iter_seek_start(struct table_iter *ti, uint8_t typ, int index)\n {\n-\tstruct reftable_reader_offsets *offs = reader_offsets_for(ti->r, typ);\n+\tstruct reftable_table_offsets *offs = table_offsets_for(ti->table, typ);\n \tuint64_t off = offs->offset;\n \tif (index) {\n \t\toff = offs->index_offset;\n@@ -518,7 +518,7 @@ static int table_iter_seek(struct table_iter *ti,\n \t\t\t   struct reftable_record *want)\n {\n \tuint8_t typ = reftable_record_type(want);\n-\tstruct reftable_reader_offsets *offs = reader_offsets_for(ti->r, typ);\n+\tstruct reftable_table_offsets *offs = table_offsets_for(ti->table, typ);\n \tint err;\n \n \terr = table_iter_seek_start(ti, reftable_record_type(want),\n@@ -566,11 +566,11 @@ static void iterator_from_table_iter(struct reftable_iterator *it,\n \tit->ops = &table_iter_vtable;\n }\n \n-int reader_init_iter(struct reftable_reader *r,\n-\t\t     struct reftable_iterator *it,\n-\t\t     uint8_t typ)\n+int table_init_iter(struct reftable_table *t,\n+\t\t    struct reftable_iterator *it,\n+\t\t    uint8_t typ)\n {\n-\tstruct reftable_reader_offsets *offs = reader_offsets_for(r, typ);\n+\tstruct reftable_table_offsets *offs = table_offsets_for(t, typ);\n \n \tif (offs->is_present) {\n \t\tstruct table_iter *ti;\n@@ -578,7 +578,7 @@ int reader_init_iter(struct reftable_reader *r,\n \t\tif (!ti)\n \t\t\treturn REFTABLE_OUT_OF_MEMORY_ERROR;\n \n-\t\ttable_iter_init(ti, r);\n+\t\ttable_iter_init(ti, t);\n \t\titerator_from_table_iter(it, ti);\n \t} else {\n \t\titerator_set_empty(it);\n@@ -587,31 +587,31 @@ int reader_init_iter(struct reftable_reader *r,\n \treturn 0;\n }\n \n-int reftable_reader_init_ref_iterator(struct reftable_reader *r,\n-\t\t\t\t      struct reftable_iterator *it)\n+int reftable_table_init_ref_iterator(struct reftable_table *t,\n+\t\t\t\t     struct reftable_iterator *it)\n {\n-\treturn reader_init_iter(r, it, BLOCK_TYPE_REF);\n+\treturn table_init_iter(t, it, BLOCK_TYPE_REF);\n }\n \n-int reftable_reader_init_log_iterator(struct reftable_reader *r,\n-\t\t\t\t      struct reftable_iterator *it)\n+int reftable_table_init_log_iterator(struct reftable_table *t,\n+\t\t\t\t     struct reftable_iterator *it)\n {\n-\treturn reader_init_iter(r, it, BLOCK_TYPE_LOG);\n+\treturn table_init_iter(t, it, BLOCK_TYPE_LOG);\n }\n \n-int reftable_reader_new(struct reftable_reader **out,\n-\t\t\tstruct reftable_block_source *source, char const *name)\n+int reftable_table_new(struct reftable_table **out,\n+\t\t       struct reftable_block_source *source, char const *name)\n {\n \tstruct reftable_block footer = { 0 };\n \tstruct reftable_block header = { 0 };\n-\tstruct reftable_reader *r;\n+\tstruct reftable_table *t;\n \tuint64_t file_size = block_source_size(source);\n \tuint32_t read_size;\n \tssize_t bytes_read;\n \tint err;\n \n-\tREFTABLE_CALLOC_ARRAY(r, 1);\n-\tif (!r) {\n+\tREFTABLE_CALLOC_ARRAY(t, 1);\n+\tif (!t) {\n \t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\tgoto done;\n \t}\n@@ -636,72 +636,72 @@ int reftable_reader_new(struct reftable_reader **out,\n \t\terr = REFTABLE_FORMAT_ERROR;\n \t\tgoto done;\n \t}\n-\tr->version = header.data[4];\n-\tif (r->version != 1 && r->version != 2) {\n+\tt->version = header.data[4];\n+\tif (t->version != 1 && t->version != 2) {\n \t\terr = REFTABLE_FORMAT_ERROR;\n \t\tgoto done;\n \t}\n \n-\tr->size = file_size - footer_size(r->version);\n-\tr->source = *source;\n-\tr->name = reftable_strdup(name);\n-\tif (!r->name) {\n+\tt->size = file_size - footer_size(t->version);\n+\tt->source = *source;\n+\tt->name = reftable_strdup(name);\n+\tif (!t->name) {\n \t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\tgoto done;\n \t}\n-\tr->hash_id = 0;\n-\tr->refcount = 1;\n+\tt->hash_id = 0;\n+\tt->refcount = 1;\n \n-\tbytes_read = block_source_read_block(source, &footer, r->size,\n-\t\t\t\t\t     footer_size(r->version));\n-\tif (bytes_read < 0 || (size_t)bytes_read != footer_size(r->version)) {\n+\tbytes_read = block_source_read_block(source, &footer, t->size,\n+\t\t\t\t\t     footer_size(t->version));\n+\tif (bytes_read < 0 || (size_t)bytes_read != footer_size(t->version)) {\n \t\terr = REFTABLE_IO_ERROR;\n \t\tgoto done;\n \t}\n \n-\terr = parse_footer(r, footer.data, header.data);\n+\terr = parse_footer(t, footer.data, header.data);\n \tif (err)\n \t\tgoto done;\n \n-\t*out = r;\n+\t*out = t;\n \n done:\n \treftable_block_done(&footer);\n \treftable_block_done(&header);\n \tif (err) {\n-\t\tif (r)\n-\t\t\treftable_free(r->name);\n-\t\treftable_free(r);\n+\t\tif (t)\n+\t\t\treftable_free(t->name);\n+\t\treftable_free(t);\n \t\tblock_source_close(source);\n \t}\n \treturn err;\n }\n \n-void reftable_reader_incref(struct reftable_reader *r)\n+void reftable_table_incref(struct reftable_table *t)\n {\n-\tr->refcount++;\n+\tt->refcount++;\n }\n \n-void reftable_reader_decref(struct reftable_reader *r)\n+void reftable_table_decref(struct reftable_table *t)\n {\n-\tif (!r)\n+\tif (!t)\n \t\treturn;\n-\tif (--r->refcount)\n+\tif (--t->refcount)\n \t\treturn;\n-\tblock_source_close(&r->source);\n-\tREFTABLE_FREE_AND_NULL(r->name);\n-\treftable_free(r);\n+\tblock_source_close(&t->source);\n+\tREFTABLE_FREE_AND_NULL(t->name);\n+\treftable_free(t);\n }\n \n-static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n-\t\t\t\t\t    struct reftable_iterator *it,\n-\t\t\t\t\t    uint8_t *oid)\n+static int reftable_table_refs_for_indexed(struct reftable_table *t,\n+\t\t\t\t\t   struct reftable_iterator *it,\n+\t\t\t\t\t   uint8_t *oid)\n {\n \tstruct reftable_record want = {\n \t\t.type = BLOCK_TYPE_OBJ,\n \t\t.u.obj = {\n \t\t\t.hash_prefix = oid,\n-\t\t\t.hash_prefix_len = r->object_id_len,\n+\t\t\t.hash_prefix_len = t->object_id_len,\n \t\t},\n \t};\n \tstruct reftable_iterator oit = { NULL };\n@@ -713,7 +713,7 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \tstruct indexed_table_ref_iter *itr = NULL;\n \n \t/* Look through the reverse index. */\n-\terr = reader_init_iter(r, &oit, BLOCK_TYPE_OBJ);\n+\terr = table_init_iter(t, &oit, BLOCK_TYPE_OBJ);\n \tif (err < 0)\n \t\tgoto done;\n \n@@ -727,14 +727,14 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \t\tgoto done;\n \n \tif (err > 0 || memcmp(want.u.obj.hash_prefix, got.u.obj.hash_prefix,\n-\t\t\t      r->object_id_len)) {\n+\t\t\t      t->object_id_len)) {\n \t\t/* didn't find it; return empty iterator */\n \t\titerator_set_empty(it);\n \t\terr = 0;\n \t\tgoto done;\n \t}\n \n-\terr = indexed_table_ref_iter_new(&itr, r, oid, hash_size(r->hash_id),\n+\terr = indexed_table_ref_iter_new(&itr, t, oid, hash_size(t->hash_id),\n \t\t\t\t\t got.u.obj.offsets,\n \t\t\t\t\t got.u.obj.offset_len);\n \tif (err < 0)\n@@ -748,14 +748,14 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \treturn err;\n }\n \n-static int reftable_reader_refs_for_unindexed(struct reftable_reader *r,\n-\t\t\t\t\t      struct reftable_iterator *it,\n-\t\t\t\t\t      uint8_t *oid)\n+static int reftable_table_refs_for_unindexed(struct reftable_table *t,\n+\t\t\t\t\t     struct reftable_iterator *it,\n+\t\t\t\t\t     uint8_t *oid)\n {\n \tstruct table_iter *ti;\n \tstruct filtering_ref_iterator *filter = NULL;\n \tstruct filtering_ref_iterator empty = FILTERING_REF_ITERATOR_INIT;\n-\tuint32_t oid_len = hash_size(r->hash_id);\n+\tuint32_t oid_len = hash_size(t->hash_id);\n \tint err;\n \n \tREFTABLE_ALLOC_ARRAY(ti, 1);\n@@ -764,7 +764,7 @@ static int reftable_reader_refs_for_unindexed(struct reftable_reader *r,\n \t\tgoto out;\n \t}\n \n-\ttable_iter_init(ti, r);\n+\ttable_iter_init(ti, t);\n \terr = table_iter_seek_start(ti, BLOCK_TYPE_REF, 0);\n \tif (err < 0)\n \t\tgoto out;\n@@ -795,25 +795,25 @@ static int reftable_reader_refs_for_unindexed(struct reftable_reader *r,\n \treturn err;\n }\n \n-int reftable_reader_refs_for(struct reftable_reader *r,\n-\t\t\t     struct reftable_iterator *it, uint8_t *oid)\n+int reftable_table_refs_for(struct reftable_table *t,\n+\t\t\t    struct reftable_iterator *it, uint8_t *oid)\n {\n-\tif (r->obj_offsets.is_present)\n-\t\treturn reftable_reader_refs_for_indexed(r, it, oid);\n-\treturn reftable_reader_refs_for_unindexed(r, it, oid);\n+\tif (t->obj_offsets.is_present)\n+\t\treturn reftable_table_refs_for_indexed(t, it, oid);\n+\treturn reftable_table_refs_for_unindexed(t, it, oid);\n }\n \n-uint64_t reftable_reader_max_update_index(struct reftable_reader *r)\n+uint64_t reftable_table_max_update_index(struct reftable_table *t)\n {\n-\treturn r->max_update_index;\n+\treturn t->max_update_index;\n }\n \n-uint64_t reftable_reader_min_update_index(struct reftable_reader *r)\n+uint64_t reftable_table_min_update_index(struct reftable_table *t)\n {\n-\treturn r->min_update_index;\n+\treturn t->min_update_index;\n }\n \n-int reftable_reader_print_blocks(const char *tablename)\n+int reftable_table_print_blocks(const char *tablename)\n {\n \tstruct {\n \t\tconst char *name;\n@@ -833,7 +833,7 @@ int reftable_reader_print_blocks(const char *tablename)\n \t\t},\n \t};\n \tstruct reftable_block_source src = { 0 };\n-\tstruct reftable_reader *r = NULL;\n+\tstruct reftable_table *table = NULL;\n \tstruct table_iter ti = { 0 };\n \tsize_t i;\n \tint err;\n@@ -842,14 +842,14 @@ int reftable_reader_print_blocks(const char *tablename)\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = reftable_reader_new(&r, &src, tablename);\n+\terr = reftable_table_new(&table, &src, tablename);\n \tif (err < 0)\n \t\tgoto done;\n \n-\ttable_iter_init(&ti, r);\n+\ttable_iter_init(&ti, table);\n \n \tprintf(\"header:\\n\");\n-\tprintf(\"  block_size: %d\\n\", r->block_size);\n+\tprintf(\"  block_size: %d\\n\", table->block_size);\n \n \tfor (i = 0; i < sizeof(sections) / sizeof(*sections); i++) {\n \t\terr = table_iter_seek_start(&ti, sections[i].type, 0);\n@@ -873,7 +873,7 @@ int reftable_reader_print_blocks(const char *tablename)\n \t}\n \n done:\n-\treftable_reader_decref(r);\n+\treftable_table_decref(table);\n \ttable_iter_close(&ti);\n \treturn err;\n }\ndiff --git a/reftable/reader.h b/reftable/table.h\nsimilarity index 65%\nrename from reftable/reader.h\nrename to reftable/table.h\nindex c9dccbd6c97..9cd8f80a207 100644\n--- a/reftable/reader.h\n+++ b/reftable/table.h\n@@ -6,13 +6,13 @@\n  * https://developers.google.com/open-source/licenses/bsd\n  */\n \n-#ifndef READER_H\n-#define READER_H\n+#ifndef TABLE_H\n+#define TABLE_H\n \n #include \"block.h\"\n #include \"record.h\"\n #include \"reftable-iterator.h\"\n-#include \"reftable-reader.h\"\n+#include \"reftable-table.h\"\n \n uint64_t block_source_size(struct reftable_block_source *source);\n \n@@ -22,14 +22,14 @@ ssize_t block_source_read_block(struct reftable_block_source *source,\n void block_source_close(struct reftable_block_source *source);\n \n /* metadata for a block type */\n-struct reftable_reader_offsets {\n+struct reftable_table_offsets {\n \tint is_present;\n \tuint64_t offset;\n \tuint64_t index_offset;\n };\n \n /* The state for reading a reftable file. */\n-struct reftable_reader {\n+struct reftable_table {\n \t/* for convenience, associate a name with the instance. */\n \tchar *name;\n \tstruct reftable_block_source source;\n@@ -47,21 +47,21 @@ struct reftable_reader {\n \tint object_id_len;\n \tint version;\n \n-\tstruct reftable_reader_offsets ref_offsets;\n-\tstruct reftable_reader_offsets obj_offsets;\n-\tstruct reftable_reader_offsets log_offsets;\n+\tstruct reftable_table_offsets ref_offsets;\n+\tstruct reftable_table_offsets obj_offsets;\n+\tstruct reftable_table_offsets log_offsets;\n \n \tuint64_t refcount;\n };\n \n-const char *reader_name(struct reftable_reader *r);\n+const char *reftable_table_name(struct reftable_table *t);\n \n-int reader_init_iter(struct reftable_reader *r,\n-\t\t     struct reftable_iterator *it,\n-\t\t     uint8_t typ);\n+int table_init_iter(struct reftable_table *t,\n+\t\t    struct reftable_iterator *it,\n+\t\t    uint8_t typ);\n \n-/* initialize a block reader to read from `r` */\n-int reader_init_block_reader(struct reftable_reader *r, struct block_reader *br,\n-\t\t\t     uint64_t next_off, uint8_t want_typ);\n+/* initialize a block reader to read from `t` */\n+int table_init_block_reader(struct reftable_table *t, struct block_reader *br,\n+\t\t\t    uint64_t next_off, uint8_t want_typ);\n \n #endif\ndiff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c\nindex 3c72ed985b3..f8f1956f4f3 100644\n--- a/t/helper/test-reftable.c\n+++ b/t/helper/test-reftable.c\n@@ -4,8 +4,8 @@\n #include \"reftable/system.h\"\n #include \"reftable/reftable-error.h\"\n #include \"reftable/reftable-merged.h\"\n-#include \"reftable/reftable-reader.h\"\n #include \"reftable/reftable-stack.h\"\n+#include \"reftable/reftable-table.h\"\n #include \"test-tool.h\"\n \n static void print_help(void)\n@@ -126,19 +126,19 @@ static int dump_reftable(const char *tablename)\n {\n \tstruct reftable_block_source src = { 0 };\n \tstruct reftable_merged_table *mt = NULL;\n-\tstruct reftable_reader *r = NULL;\n+\tstruct reftable_table *table = NULL;\n \tint err;\n \n \terr = reftable_block_source_from_file(&src, tablename);\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = reftable_reader_new(&r, &src, tablename);\n+\terr = reftable_table_new(&table, &src, tablename);\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = reftable_merged_table_new(&mt, &r, 1,\n-\t\t\t\t\treftable_reader_hash_id(r));\n+\terr = reftable_merged_table_new(&mt, &table, 1,\n+\t\t\t\t\treftable_table_hash_id(table));\n \tif (err < 0)\n \t\tgoto done;\n \n@@ -146,7 +146,7 @@ static int dump_reftable(const char *tablename)\n \n done:\n \treftable_merged_table_free(mt);\n-\treftable_reader_decref(r);\n+\treftable_table_decref(table);\n \treturn err;\n }\n \n@@ -184,7 +184,7 @@ int cmd__dump_reftable(int argc, const char **argv)\n \targ = argv[1];\n \n \tif (opt_dump_blocks) {\n-\t\terr = reftable_reader_print_blocks(arg);\n+\t\terr = reftable_table_print_blocks(arg);\n \t} else if (opt_dump_table) {\n \t\terr = dump_reftable(arg);\n \t} else if (opt_dump_stack) {\ndiff --git a/t/meson.build b/t/meson.build\nindex a59da26be3f..ded928d9e6d 100644\n--- a/t/meson.build\n+++ b/t/meson.build\n@@ -56,10 +56,10 @@ unit_test_programs = [\n   'unit-tests/t-reftable-block.c',\n   'unit-tests/t-reftable-merged.c',\n   'unit-tests/t-reftable-pq.c',\n-  'unit-tests/t-reftable-reader.c',\n   'unit-tests/t-reftable-readwrite.c',\n   'unit-tests/t-reftable-record.c',\n   'unit-tests/t-reftable-stack.c',\n+  'unit-tests/t-reftable-table.c',\n   'unit-tests/t-trailer.c',\n   'unit-tests/t-urlmatch-normalization.c',\n ]\ndiff --git a/t/unit-tests/t-reftable-merged.c b/t/unit-tests/t-reftable-merged.c\nindex 60836f80d60..fed6beb85c0 100644\n--- a/t/unit-tests/t-reftable-merged.c\n+++ b/t/unit-tests/t-reftable-merged.c\n@@ -11,7 +11,7 @@ license that can be found in the LICENSE file or at\n #include \"reftable/blocksource.h\"\n #include \"reftable/constants.h\"\n #include \"reftable/merged.h\"\n-#include \"reftable/reader.h\"\n+#include \"reftable/table.h\"\n #include \"reftable/reftable-error.h\"\n #include \"reftable/reftable-merged.h\"\n #include \"reftable/reftable-writer.h\"\n@@ -19,7 +19,7 @@ license that can be found in the LICENSE file or at\n static struct reftable_merged_table *\n merged_table_from_records(struct reftable_ref_record **refs,\n \t\t\t  struct reftable_block_source **source,\n-\t\t\t  struct reftable_reader ***readers, const size_t *sizes,\n+\t\t\t  struct reftable_table ***tables, const size_t *sizes,\n \t\t\t  struct reftable_buf *buf, const size_t n)\n {\n \tstruct reftable_merged_table *mt = NULL;\n@@ -28,8 +28,8 @@ merged_table_from_records(struct reftable_ref_record **refs,\n \t};\n \tint err;\n \n-\tREFTABLE_CALLOC_ARRAY(*readers, n);\n-\tcheck(*readers != NULL);\n+\tREFTABLE_CALLOC_ARRAY(*tables, n);\n+\tcheck(*tables != NULL);\n \tREFTABLE_CALLOC_ARRAY(*source, n);\n \tcheck(*source != NULL);\n \n@@ -37,21 +37,21 @@ merged_table_from_records(struct reftable_ref_record **refs,\n \t\tt_reftable_write_to_buf(&buf[i], refs[i], sizes[i], NULL, 0, &opts);\n \t\tblock_source_from_buf(&(*source)[i], &buf[i]);\n \n-\t\terr = reftable_reader_new(&(*readers)[i], &(*source)[i],\n-\t\t\t\t\t  \"name\");\n+\t\terr = reftable_table_new(&(*tables)[i], &(*source)[i],\n+\t\t\t\t\t \"name\");\n \t\tcheck(!err);\n \t}\n \n-\terr = reftable_merged_table_new(&mt, *readers, n, REFTABLE_HASH_SHA1);\n+\terr = reftable_merged_table_new(&mt, *tables, n, REFTABLE_HASH_SHA1);\n \tcheck(!err);\n \treturn mt;\n }\n \n-static void readers_destroy(struct reftable_reader **readers, const size_t n)\n+static void tables_destroy(struct reftable_table **tables, const size_t n)\n {\n \tfor (size_t i = 0; i < n; i++)\n-\t\treftable_reader_decref(readers[i]);\n-\treftable_free(readers);\n+\t\treftable_table_decref(tables[i]);\n+\treftable_free(tables);\n }\n \n static void t_merged_single_record(void)\n@@ -77,9 +77,9 @@ static void t_merged_single_record(void)\n \tsize_t sizes[] = { ARRAY_SIZE(r1), ARRAY_SIZE(r2), ARRAY_SIZE(r3) };\n \tstruct reftable_buf bufs[3] = { REFTABLE_BUF_INIT, REFTABLE_BUF_INIT, REFTABLE_BUF_INIT };\n \tstruct reftable_block_source *bs = NULL;\n-\tstruct reftable_reader **readers = NULL;\n+\tstruct reftable_table **tables = NULL;\n \tstruct reftable_merged_table *mt =\n-\t\tmerged_table_from_records(refs, &bs, &readers, sizes, bufs, 3);\n+\t\tmerged_table_from_records(refs, &bs, &tables, sizes, bufs, 3);\n \tstruct reftable_ref_record ref = { 0 };\n \tstruct reftable_iterator it = { 0 };\n \tint err;\n@@ -94,7 +94,7 @@ static void t_merged_single_record(void)\n \tcheck(reftable_ref_record_equal(&r2[0], &ref, REFTABLE_HASH_SIZE_SHA1));\n \treftable_ref_record_release(&ref);\n \treftable_iterator_destroy(&it);\n-\treaders_destroy(readers, 3);\n+\ttables_destroy(tables, 3);\n \treftable_merged_table_free(mt);\n \tfor (size_t i = 0; i < ARRAY_SIZE(bufs); i++)\n \t\treftable_buf_release(&bufs[i]);\n@@ -154,9 +154,9 @@ static void t_merged_refs(void)\n \tsize_t sizes[3] = { ARRAY_SIZE(r1), ARRAY_SIZE(r2), ARRAY_SIZE(r3) };\n \tstruct reftable_buf bufs[3] = { REFTABLE_BUF_INIT, REFTABLE_BUF_INIT, REFTABLE_BUF_INIT };\n \tstruct reftable_block_source *bs = NULL;\n-\tstruct reftable_reader **readers = NULL;\n+\tstruct reftable_table **tables = NULL;\n \tstruct reftable_merged_table *mt =\n-\t\tmerged_table_from_records(refs, &bs, &readers, sizes, bufs, 3);\n+\t\tmerged_table_from_records(refs, &bs, &tables, sizes, bufs, 3);\n \tstruct reftable_iterator it = { 0 };\n \tint err;\n \tstruct reftable_ref_record *out = NULL;\n@@ -193,7 +193,7 @@ static void t_merged_refs(void)\n \n \tfor (i = 0; i < 3; i++)\n \t\treftable_buf_release(&bufs[i]);\n-\treaders_destroy(readers, 3);\n+\ttables_destroy(tables, 3);\n \treftable_merged_table_free(mt);\n \treftable_free(bs);\n }\n@@ -238,12 +238,12 @@ static void t_merged_seek_multiple_times(void)\n \t\tREFTABLE_BUF_INIT, REFTABLE_BUF_INIT,\n \t};\n \tstruct reftable_block_source *sources = NULL;\n-\tstruct reftable_reader **readers = NULL;\n+\tstruct reftable_table **tables = NULL;\n \tstruct reftable_ref_record rec = { 0 };\n \tstruct reftable_iterator it = { 0 };\n \tstruct reftable_merged_table *mt;\n \n-\tmt = merged_table_from_records(refs, &sources, &readers, sizes, bufs, 2);\n+\tmt = merged_table_from_records(refs, &sources, &tables, sizes, bufs, 2);\n \tmerged_table_init_iter(mt, &it, BLOCK_TYPE_REF);\n \n \tfor (size_t i = 0; i < 5; i++) {\n@@ -266,7 +266,7 @@ static void t_merged_seek_multiple_times(void)\n \n \tfor (size_t i = 0; i < ARRAY_SIZE(bufs); i++)\n \t\treftable_buf_release(&bufs[i]);\n-\treaders_destroy(readers, ARRAY_SIZE(refs));\n+\ttables_destroy(tables, ARRAY_SIZE(refs));\n \treftable_ref_record_release(&rec);\n \treftable_iterator_destroy(&it);\n \treftable_merged_table_free(mt);\n@@ -313,13 +313,13 @@ static void t_merged_seek_multiple_times_without_draining(void)\n \t\tREFTABLE_BUF_INIT, REFTABLE_BUF_INIT,\n \t};\n \tstruct reftable_block_source *sources = NULL;\n-\tstruct reftable_reader **readers = NULL;\n+\tstruct reftable_table **tables = NULL;\n \tstruct reftable_ref_record rec = { 0 };\n \tstruct reftable_iterator it = { 0 };\n \tstruct reftable_merged_table *mt;\n \tint err;\n \n-\tmt = merged_table_from_records(refs, &sources, &readers, sizes, bufs, 2);\n+\tmt = merged_table_from_records(refs, &sources, &tables, sizes, bufs, 2);\n \tmerged_table_init_iter(mt, &it, BLOCK_TYPE_REF);\n \n \terr = reftable_iterator_seek_ref(&it, \"b\");\n@@ -338,7 +338,7 @@ static void t_merged_seek_multiple_times_without_draining(void)\n \n \tfor (size_t i = 0; i < ARRAY_SIZE(bufs); i++)\n \t\treftable_buf_release(&bufs[i]);\n-\treaders_destroy(readers, ARRAY_SIZE(refs));\n+\ttables_destroy(tables, ARRAY_SIZE(refs));\n \treftable_ref_record_release(&rec);\n \treftable_iterator_destroy(&it);\n \treftable_merged_table_free(mt);\n@@ -348,7 +348,7 @@ static void t_merged_seek_multiple_times_without_draining(void)\n static struct reftable_merged_table *\n merged_table_from_log_records(struct reftable_log_record **logs,\n \t\t\t      struct reftable_block_source **source,\n-\t\t\t      struct reftable_reader ***readers, const size_t *sizes,\n+\t\t\t      struct reftable_table ***tables, const size_t *sizes,\n \t\t\t      struct reftable_buf *buf, const size_t n)\n {\n \tstruct reftable_merged_table *mt = NULL;\n@@ -358,8 +358,8 @@ merged_table_from_log_records(struct reftable_log_record **logs,\n \t};\n \tint err;\n \n-\tREFTABLE_CALLOC_ARRAY(*readers, n);\n-\tcheck(*readers != NULL);\n+\tREFTABLE_CALLOC_ARRAY(*tables, n);\n+\tcheck(*tables != NULL);\n \tREFTABLE_CALLOC_ARRAY(*source, n);\n \tcheck(*source != NULL);\n \n@@ -367,12 +367,12 @@ merged_table_from_log_records(struct reftable_log_record **logs,\n \t\tt_reftable_write_to_buf(&buf[i], NULL, 0, logs[i], sizes[i], &opts);\n \t\tblock_source_from_buf(&(*source)[i], &buf[i]);\n \n-\t\terr = reftable_reader_new(&(*readers)[i], &(*source)[i],\n-\t\t\t\t\t  \"name\");\n+\t\terr = reftable_table_new(&(*tables)[i], &(*source)[i],\n+\t\t\t\t\t \"name\");\n \t\tcheck(!err);\n \t}\n \n-\terr = reftable_merged_table_new(&mt, *readers, n, REFTABLE_HASH_SHA1);\n+\terr = reftable_merged_table_new(&mt, *tables, n, REFTABLE_HASH_SHA1);\n \tcheck(!err);\n \treturn mt;\n }\n@@ -435,9 +435,9 @@ static void t_merged_logs(void)\n \tsize_t sizes[3] = { ARRAY_SIZE(r1), ARRAY_SIZE(r2), ARRAY_SIZE(r3) };\n \tstruct reftable_buf bufs[3] = { REFTABLE_BUF_INIT, REFTABLE_BUF_INIT, REFTABLE_BUF_INIT };\n \tstruct reftable_block_source *bs = NULL;\n-\tstruct reftable_reader **readers = NULL;\n+\tstruct reftable_table **tables = NULL;\n \tstruct reftable_merged_table *mt = merged_table_from_log_records(\n-\t\tlogs, &bs, &readers, sizes, bufs, 3);\n+\t\tlogs, &bs, &tables, sizes, bufs, 3);\n \tstruct reftable_iterator it = { 0 };\n \tint err;\n \tstruct reftable_log_record *out = NULL;\n@@ -485,7 +485,7 @@ static void t_merged_logs(void)\n \n \tfor (i = 0; i < 3; i++)\n \t\treftable_buf_release(&bufs[i]);\n-\treaders_destroy(readers, 3);\n+\ttables_destroy(tables, 3);\n \treftable_merged_table_free(mt);\n \treftable_free(bs);\n }\n@@ -502,7 +502,7 @@ static void t_default_write_opts(void)\n \tint err;\n \tstruct reftable_block_source source = { 0 };\n \tuint32_t hash_id;\n-\tstruct reftable_reader *rd = NULL;\n+\tstruct reftable_table *table = NULL;\n \tstruct reftable_merged_table *merged = NULL;\n \n \treftable_writer_set_limits(w, 1, 1);\n@@ -516,18 +516,18 @@ static void t_default_write_opts(void)\n \n \tblock_source_from_buf(&source, &buf);\n \n-\terr = reftable_reader_new(&rd, &source, \"filename\");\n+\terr = reftable_table_new(&table, &source, \"filename\");\n \tcheck(!err);\n \n-\thash_id = reftable_reader_hash_id(rd);\n+\thash_id = reftable_table_hash_id(table);\n \tcheck_int(hash_id, ==, REFTABLE_HASH_SHA1);\n \n-\terr = reftable_merged_table_new(&merged, &rd, 1, REFTABLE_HASH_SHA256);\n+\terr = reftable_merged_table_new(&merged, &table, 1, REFTABLE_HASH_SHA256);\n \tcheck_int(err, ==, REFTABLE_FORMAT_ERROR);\n-\terr = reftable_merged_table_new(&merged, &rd, 1, REFTABLE_HASH_SHA1);\n+\terr = reftable_merged_table_new(&merged, &table, 1, REFTABLE_HASH_SHA1);\n \tcheck(!err);\n \n-\treftable_reader_decref(rd);\n+\treftable_table_decref(table);\n \treftable_merged_table_free(merged);\n \treftable_buf_release(&buf);\n }\ndiff --git a/t/unit-tests/t-reftable-readwrite.c b/t/unit-tests/t-reftable-readwrite.c\nindex c9626831dac..c4c27242ba9 100644\n--- a/t/unit-tests/t-reftable-readwrite.c\n+++ b/t/unit-tests/t-reftable-readwrite.c\n@@ -12,9 +12,9 @@ license that can be found in the LICENSE file or at\n #include \"lib-reftable.h\"\n #include \"reftable/basics.h\"\n #include \"reftable/blocksource.h\"\n-#include \"reftable/reader.h\"\n #include \"reftable/reftable-error.h\"\n #include \"reftable/reftable-writer.h\"\n+#include \"reftable/table.h\"\n #include \"strbuf.h\"\n \n static const int update_index = 5;\n@@ -204,7 +204,7 @@ static void t_log_write_read(void)\n \tstruct reftable_ref_record ref = { 0 };\n \tstruct reftable_log_record log = { 0 };\n \tstruct reftable_iterator it = { 0 };\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tstruct reftable_block_source source = { 0 };\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tstruct reftable_writer *w = t_reftable_strbuf_writer(&buf, &opts);\n@@ -254,10 +254,10 @@ static void t_log_write_read(void)\n \n \tblock_source_from_buf(&source, &buf);\n \n-\terr = reftable_reader_new(&reader, &source, \"file.log\");\n+\terr = reftable_table_new(&table, &source, \"file.log\");\n \tcheck(!err);\n \n-\terr = reftable_reader_init_ref_iterator(reader, &it);\n+\terr = reftable_table_init_ref_iterator(table, &it);\n \tcheck(!err);\n \n \terr = reftable_iterator_seek_ref(&it, names[N - 1]);\n@@ -273,7 +273,7 @@ static void t_log_write_read(void)\n \treftable_iterator_destroy(&it);\n \treftable_ref_record_release(&ref);\n \n-\terr = reftable_reader_init_log_iterator(reader, &it);\n+\terr = reftable_table_init_log_iterator(table, &it);\n \tcheck(!err);\n \terr = reftable_iterator_seek_log(&it, \"\");\n \tcheck(!err);\n@@ -294,7 +294,7 @@ static void t_log_write_read(void)\n \t/* cleanup. */\n \treftable_buf_release(&buf);\n \tfree_names(names);\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n }\n \n static void t_log_zlib_corruption(void)\n@@ -303,7 +303,7 @@ static void t_log_zlib_corruption(void)\n \t\t.block_size = 256,\n \t};\n \tstruct reftable_iterator it = { 0 };\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tstruct reftable_block_source source = { 0 };\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tstruct reftable_writer *w = t_reftable_strbuf_writer(&buf, &opts);\n@@ -345,10 +345,10 @@ static void t_log_zlib_corruption(void)\n \n \tblock_source_from_buf(&source, &buf);\n \n-\terr = reftable_reader_new(&reader, &source, \"file.log\");\n+\terr = reftable_table_new(&table, &source, \"file.log\");\n \tcheck(!err);\n \n-\terr = reftable_reader_init_log_iterator(reader, &it);\n+\terr = reftable_table_init_log_iterator(table, &it);\n \tcheck(!err);\n \terr = reftable_iterator_seek_log(&it, \"refname\");\n \tcheck_int(err, ==, REFTABLE_ZLIB_ERROR);\n@@ -356,7 +356,7 @@ static void t_log_zlib_corruption(void)\n \treftable_iterator_destroy(&it);\n \n \t/* cleanup. */\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n \treftable_buf_release(&buf);\n }\n \n@@ -367,7 +367,7 @@ static void t_table_read_write_sequential(void)\n \tint N = 50;\n \tstruct reftable_iterator it = { 0 };\n \tstruct reftable_block_source source = { 0 };\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tint err = 0;\n \tint j = 0;\n \n@@ -375,10 +375,10 @@ static void t_table_read_write_sequential(void)\n \n \tblock_source_from_buf(&source, &buf);\n \n-\terr = reftable_reader_new(&reader, &source, \"file.ref\");\n+\terr = reftable_table_new(&table, &source, \"file.ref\");\n \tcheck(!err);\n \n-\terr = reftable_reader_init_ref_iterator(reader, &it);\n+\terr = reftable_table_init_ref_iterator(table, &it);\n \tcheck(!err);\n \terr = reftable_iterator_seek_ref(&it, \"\");\n \tcheck(!err);\n@@ -396,7 +396,7 @@ static void t_table_read_write_sequential(void)\n \tcheck_int(j, ==, N);\n \n \treftable_iterator_destroy(&it);\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n \treftable_buf_release(&buf);\n \tfree_names(names);\n }\n@@ -417,7 +417,7 @@ static void t_table_read_api(void)\n \tchar **names;\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tint N = 50;\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tstruct reftable_block_source source = { 0 };\n \tint err;\n \tstruct reftable_log_record log = { 0 };\n@@ -427,10 +427,10 @@ static void t_table_read_api(void)\n \n \tblock_source_from_buf(&source, &buf);\n \n-\terr = reftable_reader_new(&reader, &source, \"file.ref\");\n+\terr = reftable_table_new(&table, &source, \"file.ref\");\n \tcheck(!err);\n \n-\terr = reftable_reader_init_ref_iterator(reader, &it);\n+\terr = reftable_table_init_ref_iterator(table, &it);\n \tcheck(!err);\n \terr = reftable_iterator_seek_ref(&it, names[0]);\n \tcheck(!err);\n@@ -441,7 +441,7 @@ static void t_table_read_api(void)\n \treftable_buf_release(&buf);\n \tfree_names(names);\n \treftable_iterator_destroy(&it);\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n \treftable_buf_release(&buf);\n }\n \n@@ -450,7 +450,7 @@ static void t_table_read_write_seek(int index, enum reftable_hash hash_id)\n \tchar **names;\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tint N = 50;\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tstruct reftable_block_source source = { 0 };\n \tint err;\n \tint i = 0;\n@@ -463,18 +463,18 @@ static void t_table_read_write_seek(int index, enum reftable_hash hash_id)\n \n \tblock_source_from_buf(&source, &buf);\n \n-\terr = reftable_reader_new(&reader, &source, \"file.ref\");\n+\terr = reftable_table_new(&table, &source, \"file.ref\");\n \tcheck(!err);\n-\tcheck_int(hash_id, ==, reftable_reader_hash_id(reader));\n+\tcheck_int(hash_id, ==, reftable_table_hash_id(table));\n \n \tif (!index) {\n-\t\treader->ref_offsets.index_offset = 0;\n+\t\ttable->ref_offsets.index_offset = 0;\n \t} else {\n-\t\tcheck_int(reader->ref_offsets.index_offset, >, 0);\n+\t\tcheck_int(table->ref_offsets.index_offset, >, 0);\n \t}\n \n \tfor (i = 1; i < N; i++) {\n-\t\terr = reftable_reader_init_ref_iterator(reader, &it);\n+\t\terr = reftable_table_init_ref_iterator(table, &it);\n \t\tcheck(!err);\n \t\terr = reftable_iterator_seek_ref(&it, names[i]);\n \t\tcheck(!err);\n@@ -491,7 +491,7 @@ static void t_table_read_write_seek(int index, enum reftable_hash hash_id)\n \tcheck(!reftable_buf_addstr(&pastLast, names[N - 1]));\n \tcheck(!reftable_buf_addstr(&pastLast, \"/\"));\n \n-\terr = reftable_reader_init_ref_iterator(reader, &it);\n+\terr = reftable_table_init_ref_iterator(table, &it);\n \tcheck(!err);\n \terr = reftable_iterator_seek_ref(&it, pastLast.buf);\n \tif (err == 0) {\n@@ -507,7 +507,7 @@ static void t_table_read_write_seek(int index, enum reftable_hash hash_id)\n \n \treftable_buf_release(&buf);\n \tfree_names(names);\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n }\n \n static void t_table_read_write_seek_linear(void)\n@@ -535,7 +535,7 @@ static void t_table_refs_for(int indexed)\n \t\t.block_size = 256,\n \t};\n \tstruct reftable_ref_record ref = { 0 };\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tstruct reftable_block_source source = { 0 };\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tstruct reftable_writer *w = t_reftable_strbuf_writer(&buf, &opts);\n@@ -585,18 +585,18 @@ static void t_table_refs_for(int indexed)\n \n \tblock_source_from_buf(&source, &buf);\n \n-\terr = reftable_reader_new(&reader, &source, \"file.ref\");\n+\terr = reftable_table_new(&table, &source, \"file.ref\");\n \tcheck(!err);\n \tif (!indexed)\n-\t\treader->obj_offsets.is_present = 0;\n+\t\ttable->obj_offsets.is_present = 0;\n \n-\terr = reftable_reader_init_ref_iterator(reader, &it);\n+\terr = reftable_table_init_ref_iterator(table, &it);\n \tcheck(!err);\n \terr = reftable_iterator_seek_ref(&it, \"\");\n \tcheck(!err);\n \treftable_iterator_destroy(&it);\n \n-\terr = reftable_reader_refs_for(reader, &it, want_hash);\n+\terr = reftable_table_refs_for(table, &it, want_hash);\n \tcheck(!err);\n \n \tfor (j = 0; ; j++) {\n@@ -613,7 +613,7 @@ static void t_table_refs_for(int indexed)\n \treftable_buf_release(&buf);\n \tfree_names(want_names);\n \treftable_iterator_destroy(&it);\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n }\n \n static void t_table_refs_for_no_index(void)\n@@ -632,7 +632,7 @@ static void t_write_empty_table(void)\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tstruct reftable_writer *w = t_reftable_strbuf_writer(&buf, &opts);\n \tstruct reftable_block_source source = { 0 };\n-\tstruct reftable_reader *rd = NULL;\n+\tstruct reftable_table *table = NULL;\n \tstruct reftable_ref_record rec = { 0 };\n \tstruct reftable_iterator it = { 0 };\n \tint err;\n@@ -647,10 +647,10 @@ static void t_write_empty_table(void)\n \n \tblock_source_from_buf(&source, &buf);\n \n-\terr = reftable_reader_new(&rd, &source, \"filename\");\n+\terr = reftable_table_new(&table, &source, \"filename\");\n \tcheck(!err);\n \n-\terr = reftable_reader_init_ref_iterator(rd, &it);\n+\terr = reftable_table_init_ref_iterator(table, &it);\n \tcheck(!err);\n \terr = reftable_iterator_seek_ref(&it, \"\");\n \tcheck(!err);\n@@ -659,7 +659,7 @@ static void t_write_empty_table(void)\n \tcheck_int(err, >, 0);\n \n \treftable_iterator_destroy(&it);\n-\treftable_reader_decref(rd);\n+\treftable_table_decref(table);\n \treftable_buf_release(&buf);\n }\n \n@@ -803,7 +803,7 @@ static void t_write_multiple_indices(void)\n \tstruct reftable_iterator it = { 0 };\n \tconst struct reftable_stats *stats;\n \tstruct reftable_writer *writer;\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tchar buf[128];\n \tint err, i;\n \n@@ -852,21 +852,21 @@ static void t_write_multiple_indices(void)\n \tcheck_int(stats->log_stats.index_offset, >, 0);\n \n \tblock_source_from_buf(&source, &writer_buf);\n-\terr = reftable_reader_new(&reader, &source, \"filename\");\n+\terr = reftable_table_new(&table, &source, \"filename\");\n \tcheck(!err);\n \n \t/*\n \t * Seeking the log uses the log index now. In case there is any\n \t * confusion regarding indices we would notice here.\n \t */\n-\terr = reftable_reader_init_log_iterator(reader, &it);\n+\terr = reftable_table_init_log_iterator(table, &it);\n \tcheck(!err);\n \terr = reftable_iterator_seek_log(&it, \"\");\n \tcheck(!err);\n \n \treftable_iterator_destroy(&it);\n \treftable_writer_free(writer);\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n \treftable_buf_release(&writer_buf);\n }\n \n@@ -880,7 +880,7 @@ static void t_write_multi_level_index(void)\n \tstruct reftable_iterator it = { 0 };\n \tconst struct reftable_stats *stats;\n \tstruct reftable_writer *writer;\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tint err;\n \n \twriter = t_reftable_strbuf_writer(&writer_buf, &opts);\n@@ -909,20 +909,20 @@ static void t_write_multi_level_index(void)\n \tcheck_int(stats->ref_stats.max_index_level, ==, 2);\n \n \tblock_source_from_buf(&source, &writer_buf);\n-\terr = reftable_reader_new(&reader, &source, \"filename\");\n+\terr = reftable_table_new(&table, &source, \"filename\");\n \tcheck(!err);\n \n \t/*\n \t * Seeking the last ref should work as expected.\n \t */\n-\terr = reftable_reader_init_ref_iterator(reader, &it);\n+\terr = reftable_table_init_ref_iterator(table, &it);\n \tcheck(!err);\n \terr = reftable_iterator_seek_ref(&it, \"refs/heads/199\");\n \tcheck(!err);\n \n \treftable_iterator_destroy(&it);\n \treftable_writer_free(writer);\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n \treftable_buf_release(&writer_buf);\n \treftable_buf_release(&buf);\n }\n@@ -931,11 +931,11 @@ static void t_corrupt_table_empty(void)\n {\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tstruct reftable_block_source source = { 0 };\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tint err;\n \n \tblock_source_from_buf(&source, &buf);\n-\terr = reftable_reader_new(&reader, &source, \"file.log\");\n+\terr = reftable_table_new(&table, &source, \"file.log\");\n \tcheck_int(err, ==, REFTABLE_FORMAT_ERROR);\n }\n \n@@ -944,12 +944,12 @@ static void t_corrupt_table(void)\n \tuint8_t zeros[1024] = { 0 };\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tstruct reftable_block_source source = { 0 };\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tint err;\n \tcheck(!reftable_buf_add(&buf, zeros, sizeof(zeros)));\n \n \tblock_source_from_buf(&source, &buf);\n-\terr = reftable_reader_new(&reader, &source, \"file.log\");\n+\terr = reftable_table_new(&table, &source, \"file.log\");\n \tcheck_int(err, ==, REFTABLE_FORMAT_ERROR);\n \n \treftable_buf_release(&buf);\ndiff --git a/t/unit-tests/t-reftable-stack.c b/t/unit-tests/t-reftable-stack.c\nindex c3f0059c346..2f49c975194 100644\n--- a/t/unit-tests/t-reftable-stack.c\n+++ b/t/unit-tests/t-reftable-stack.c\n@@ -12,9 +12,9 @@ license that can be found in the LICENSE file or at\n #include \"lib-reftable.h\"\n #include \"dir.h\"\n #include \"reftable/merged.h\"\n-#include \"reftable/reader.h\"\n #include \"reftable/reftable-error.h\"\n #include \"reftable/stack.h\"\n+#include \"reftable/table.h\"\n #include \"strbuf.h\"\n #include \"tempfile.h\"\n #include <dirent.h>\n@@ -176,7 +176,7 @@ static void t_reftable_stack_add_one(void)\n \terr = reftable_stack_read_ref(st, ref.refname, &dest);\n \tcheck(!err);\n \tcheck(reftable_ref_record_equal(&ref, &dest, REFTABLE_HASH_SIZE_SHA1));\n-\tcheck_int(st->readers_len, >, 0);\n+\tcheck_int(st->tables_len, >, 0);\n \n #ifndef GIT_WINDOWS_NATIVE\n \tcheck(!reftable_buf_addstr(&scratch, dir));\n@@ -189,7 +189,7 @@ static void t_reftable_stack_add_one(void)\n \tcheck(!reftable_buf_addstr(&scratch, dir));\n \tcheck(!reftable_buf_addstr(&scratch, \"/\"));\n \t/* do not try at home; not an external API for reftable. */\n-\tcheck(!reftable_buf_addstr(&scratch, st->readers[0]->name));\n+\tcheck(!reftable_buf_addstr(&scratch, st->tables[0]->name));\n \terr = stat(scratch.buf, &stat_result);\n \tcheck(!err);\n \tcheck_int((stat_result.st_mode & 0777), ==, opts.default_permissions);\n@@ -402,9 +402,9 @@ static void t_reftable_stack_transaction_api_performs_auto_compaction(void)\n \t\t * all tables in the stack.\n \t\t */\n \t\tif (i != n)\n-\t\t\tcheck_int(st->merged->readers_len, ==, i + 1);\n+\t\t\tcheck_int(st->merged->tables_len, ==, i + 1);\n \t\telse\n-\t\t\tcheck_int(st->merged->readers_len, ==, 1);\n+\t\t\tcheck_int(st->merged->tables_len, ==, 1);\n \t}\n \n \treftable_stack_destroy(st);\n@@ -430,7 +430,7 @@ static void t_reftable_stack_auto_compaction_fails_gracefully(void)\n \n \terr = reftable_stack_add(st, write_test_ref, &ref);\n \tcheck(!err);\n-\tcheck_int(st->merged->readers_len, ==, 1);\n+\tcheck_int(st->merged->tables_len, ==, 1);\n \tcheck_int(st->stats.attempts, ==, 0);\n \tcheck_int(st->stats.failures, ==, 0);\n \n@@ -441,14 +441,14 @@ static void t_reftable_stack_auto_compaction_fails_gracefully(void)\n \t */\n \tcheck(!reftable_buf_addstr(&table_path, dir));\n \tcheck(!reftable_buf_addstr(&table_path, \"/\"));\n-\tcheck(!reftable_buf_addstr(&table_path, st->readers[0]->name));\n+\tcheck(!reftable_buf_addstr(&table_path, st->tables[0]->name));\n \tcheck(!reftable_buf_addstr(&table_path, \".lock\"));\n \twrite_file_buf(table_path.buf, \"\", 0);\n \n \tref.update_index = 2;\n \terr = reftable_stack_add(st, write_test_ref, &ref);\n \tcheck(!err);\n-\tcheck_int(st->merged->readers_len, ==, 2);\n+\tcheck_int(st->merged->tables_len, ==, 2);\n \tcheck_int(st->stats.attempts, ==, 1);\n \tcheck_int(st->stats.failures, ==, 1);\n \n@@ -592,7 +592,7 @@ static void t_reftable_stack_add(void)\n \tcheck(!reftable_buf_addstr(&path, dir));\n \tcheck(!reftable_buf_addstr(&path, \"/\"));\n \t/* do not try at home; not an external API for reftable. */\n-\tcheck(!reftable_buf_addstr(&path, st->readers[0]->name));\n+\tcheck(!reftable_buf_addstr(&path, st->tables[0]->name));\n \terr = stat(path.buf, &stat_result);\n \tcheck(!err);\n \tcheck_int((stat_result.st_mode & 0777), ==, opts.default_permissions);\n@@ -1026,7 +1026,7 @@ static void t_reftable_stack_auto_compaction(void)\n \n \t\terr = reftable_stack_auto_compact(st);\n \t\tcheck(!err);\n-\t\tcheck(i < 2 || st->merged->readers_len < 2 * fastlogN(i, 2));\n+\t\tcheck(i < 2 || st->merged->tables_len < 2 * fastlogN(i, 2));\n \t}\n \n \tcheck_int(reftable_stack_compaction_stats(st)->entries_written, <,\n@@ -1061,7 +1061,7 @@ static void t_reftable_stack_auto_compaction_factor(void)\n \t\terr = reftable_stack_add(st, &write_test_ref, &ref);\n \t\tcheck(!err);\n \n-\t\tcheck(i < 5 || st->merged->readers_len < 5 * fastlogN(i, 5));\n+\t\tcheck(i < 5 || st->merged->tables_len < 5 * fastlogN(i, 5));\n \t}\n \n \treftable_stack_destroy(st);\n@@ -1082,7 +1082,7 @@ static void t_reftable_stack_auto_compaction_with_locked_tables(void)\n \tcheck(!err);\n \n \twrite_n_ref_tables(st, 5);\n-\tcheck_int(st->merged->readers_len, ==, 5);\n+\tcheck_int(st->merged->tables_len, ==, 5);\n \n \t/*\n \t * Given that all tables we have written should be roughly the same\n@@ -1091,7 +1091,7 @@ static void t_reftable_stack_auto_compaction_with_locked_tables(void)\n \t */\n \tcheck(!reftable_buf_addstr(&buf, dir));\n \tcheck(!reftable_buf_addstr(&buf, \"/\"));\n-\tcheck(!reftable_buf_addstr(&buf, st->readers[2]->name));\n+\tcheck(!reftable_buf_addstr(&buf, st->tables[2]->name));\n \tcheck(!reftable_buf_addstr(&buf, \".lock\"));\n \twrite_file_buf(buf.buf, \"\", 0);\n \n@@ -1104,7 +1104,7 @@ static void t_reftable_stack_auto_compaction_with_locked_tables(void)\n \terr = reftable_stack_auto_compact(st);\n \tcheck(!err);\n \tcheck_int(st->stats.failures, ==, 0);\n-\tcheck_int(st->merged->readers_len, ==, 4);\n+\tcheck_int(st->merged->tables_len, ==, 4);\n \n \treftable_stack_destroy(st);\n \treftable_buf_release(&buf);\n@@ -1149,9 +1149,9 @@ static void t_reftable_stack_add_performs_auto_compaction(void)\n \t\t * all tables in the stack.\n \t\t */\n \t\tif (i != n)\n-\t\t\tcheck_int(st->merged->readers_len, ==, i + 1);\n+\t\t\tcheck_int(st->merged->tables_len, ==, i + 1);\n \t\telse\n-\t\t\tcheck_int(st->merged->readers_len, ==, 1);\n+\t\t\tcheck_int(st->merged->tables_len, ==, 1);\n \t}\n \n \treftable_stack_destroy(st);\n@@ -1172,12 +1172,12 @@ static void t_reftable_stack_compaction_with_locked_tables(void)\n \tcheck(!err);\n \n \twrite_n_ref_tables(st, 3);\n-\tcheck_int(st->merged->readers_len, ==, 3);\n+\tcheck_int(st->merged->tables_len, ==, 3);\n \n \t/* Lock one of the tables that we're about to compact. */\n \tcheck(!reftable_buf_addstr(&buf, dir));\n \tcheck(!reftable_buf_addstr(&buf, \"/\"));\n-\tcheck(!reftable_buf_addstr(&buf, st->readers[1]->name));\n+\tcheck(!reftable_buf_addstr(&buf, st->tables[1]->name));\n \tcheck(!reftable_buf_addstr(&buf, \".lock\"));\n \twrite_file_buf(buf.buf, \"\", 0);\n \n@@ -1188,7 +1188,7 @@ static void t_reftable_stack_compaction_with_locked_tables(void)\n \terr = reftable_stack_compact_all(st, NULL);\n \tcheck_int(err, ==, REFTABLE_LOCK_ERROR);\n \tcheck_int(st->stats.failures, ==, 1);\n-\tcheck_int(st->merged->readers_len, ==, 3);\n+\tcheck_int(st->merged->tables_len, ==, 3);\n \n \treftable_stack_destroy(st);\n \treftable_buf_release(&buf);\n@@ -1222,10 +1222,10 @@ static void t_reftable_stack_compaction_concurrent(void)\n static void unclean_stack_close(struct reftable_stack *st)\n {\n \t/* break abstraction boundary to simulate unclean shutdown. */\n-\tfor (size_t i = 0; i < st->readers_len; i++)\n-\t\treftable_reader_decref(st->readers[i]);\n-\tst->readers_len = 0;\n-\tREFTABLE_FREE_AND_NULL(st->readers);\n+\tfor (size_t i = 0; i < st->tables_len; i++)\n+\t\treftable_table_decref(st->tables[i]);\n+\tst->tables_len = 0;\n+\tREFTABLE_FREE_AND_NULL(st->tables);\n }\n \n static void t_reftable_stack_compaction_concurrent_clean(void)\n@@ -1275,7 +1275,7 @@ static void t_reftable_stack_read_across_reload(void)\n \terr = reftable_new_stack(&st1, dir, &opts);\n \tcheck(!err);\n \twrite_n_ref_tables(st1, 2);\n-\tcheck_int(st1->merged->readers_len, ==, 2);\n+\tcheck_int(st1->merged->tables_len, ==, 2);\n \treftable_stack_init_ref_iterator(st1, &it);\n \terr = reftable_iterator_seek_ref(&it, \"\");\n \tcheck(!err);\n@@ -1283,10 +1283,10 @@ static void t_reftable_stack_read_across_reload(void)\n \t/* Set up a second stack for the same directory and compact it. */\n \terr = reftable_new_stack(&st2, dir, &opts);\n \tcheck(!err);\n-\tcheck_int(st2->merged->readers_len, ==, 2);\n+\tcheck_int(st2->merged->tables_len, ==, 2);\n \terr = reftable_stack_compact_all(st2, NULL);\n \tcheck(!err);\n-\tcheck_int(st2->merged->readers_len, ==, 1);\n+\tcheck_int(st2->merged->tables_len, ==, 1);\n \n \t/*\n \t * Verify that we can continue to use the old iterator even after we\n@@ -1294,7 +1294,7 @@ static void t_reftable_stack_read_across_reload(void)\n \t */\n \terr = reftable_stack_reload(st1);\n \tcheck(!err);\n-\tcheck_int(st1->merged->readers_len, ==, 1);\n+\tcheck_int(st1->merged->tables_len, ==, 1);\n \terr = reftable_iterator_next_ref(&it, &rec);\n \tcheck(!err);\n \tcheck_str(rec.refname, \"refs/heads/branch-0000\");\n@@ -1325,19 +1325,19 @@ static void t_reftable_stack_reload_with_missing_table(void)\n \terr = reftable_new_stack(&st, dir, &opts);\n \tcheck(!err);\n \twrite_n_ref_tables(st, 2);\n-\tcheck_int(st->merged->readers_len, ==, 2);\n+\tcheck_int(st->merged->tables_len, ==, 2);\n \treftable_stack_init_ref_iterator(st, &it);\n \terr = reftable_iterator_seek_ref(&it, \"\");\n \tcheck(!err);\n \n \t/*\n \t * Update the tables.list file with some garbage data, while reusing\n-\t * our old readers. This should trigger a partial reload of the stack,\n-\t * where we try to reuse our old readers.\n+\t * our old tables. This should trigger a partial reload of the stack,\n+\t * where we try to reuse our old tables.\n \t*/\n-\tcheck(!reftable_buf_addstr(&content, st->readers[0]->name));\n+\tcheck(!reftable_buf_addstr(&content, st->tables[0]->name));\n \tcheck(!reftable_buf_addstr(&content, \"\\n\"));\n-\tcheck(!reftable_buf_addstr(&content, st->readers[1]->name));\n+\tcheck(!reftable_buf_addstr(&content, st->tables[1]->name));\n \tcheck(!reftable_buf_addstr(&content, \"\\n\"));\n \tcheck(!reftable_buf_addstr(&content, \"garbage\\n\"));\n \tcheck(!reftable_buf_addstr(&table_path, st->list_file));\n@@ -1348,7 +1348,7 @@ static void t_reftable_stack_reload_with_missing_table(void)\n \n \terr = reftable_stack_reload(st);\n \tcheck_int(err, ==, -4);\n-\tcheck_int(st->merged->readers_len, ==, 2);\n+\tcheck_int(st->merged->tables_len, ==, 2);\n \n \t/*\n \t * Even though the reload has failed, we should be able to continue\ndiff --git a/t/unit-tests/t-reftable-reader.c b/t/unit-tests/t-reftable-table.c\nsimilarity index 78%\nrename from t/unit-tests/t-reftable-reader.c\nrename to t/unit-tests/t-reftable-table.c\nindex 546df6005e4..77c59dbf46d 100644\n--- a/t/unit-tests/t-reftable-reader.c\n+++ b/t/unit-tests/t-reftable-table.c\n@@ -1,9 +1,9 @@\n #include \"test-lib.h\"\n #include \"lib-reftable.h\"\n #include \"reftable/blocksource.h\"\n-#include \"reftable/reader.h\"\n+#include \"reftable/table.h\"\n \n-static int t_reader_seek_once(void)\n+static int t_table_seek_once(void)\n {\n \tstruct reftable_ref_record records[] = {\n \t\t{\n@@ -15,17 +15,17 @@ static int t_reader_seek_once(void)\n \tstruct reftable_block_source source = { 0 };\n \tstruct reftable_ref_record ref = { 0 };\n \tstruct reftable_iterator it = { 0 };\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tint ret;\n \n \tt_reftable_write_to_buf(&buf, records, ARRAY_SIZE(records), NULL, 0, NULL);\n \tblock_source_from_buf(&source, &buf);\n \n-\tret = reftable_reader_new(&reader, &source, \"name\");\n+\tret = reftable_table_new(&table, &source, \"name\");\n \tcheck(!ret);\n \n-\treftable_reader_init_ref_iterator(reader, &it);\n+\treftable_table_init_ref_iterator(table, &it);\n \tret = reftable_iterator_seek_ref(&it, \"\");\n \tcheck(!ret);\n \tret = reftable_iterator_next_ref(&it, &ref);\n@@ -39,12 +39,12 @@ static int t_reader_seek_once(void)\n \n \treftable_ref_record_release(&ref);\n \treftable_iterator_destroy(&it);\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n \treftable_buf_release(&buf);\n \treturn 0;\n }\n \n-static int t_reader_reseek(void)\n+static int t_table_reseek(void)\n {\n \tstruct reftable_ref_record records[] = {\n \t\t{\n@@ -56,17 +56,17 @@ static int t_reader_reseek(void)\n \tstruct reftable_block_source source = { 0 };\n \tstruct reftable_ref_record ref = { 0 };\n \tstruct reftable_iterator it = { 0 };\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tint ret;\n \n \tt_reftable_write_to_buf(&buf, records, ARRAY_SIZE(records), NULL, 0, NULL);\n \tblock_source_from_buf(&source, &buf);\n \n-\tret = reftable_reader_new(&reader, &source, \"name\");\n+\tret = reftable_table_new(&table, &source, \"name\");\n \tcheck(!ret);\n \n-\treftable_reader_init_ref_iterator(reader, &it);\n+\treftable_table_init_ref_iterator(table, &it);\n \n \tfor (size_t i = 0; i < 5; i++) {\n \t\tret = reftable_iterator_seek_ref(&it, \"\");\n@@ -83,14 +83,14 @@ static int t_reader_reseek(void)\n \n \treftable_ref_record_release(&ref);\n \treftable_iterator_destroy(&it);\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n \treftable_buf_release(&buf);\n \treturn 0;\n }\n \n int cmd_main(int argc UNUSED, const char *argv[] UNUSED)\n {\n-\tTEST(t_reader_seek_once(), \"reader can seek once\");\n-\tTEST(t_reader_reseek(), \"reader can reseek multiple times\");\n+\tTEST(t_table_seek_once(), \"table can seek once\");\n+\tTEST(t_table_reseek(), \"table can reseek multiple times\");\n \treturn test_done();\n }\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515341","messageId":"20250331-pks-reftable-polishing-v1-4-ebed5247434c@pks.im","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im","subject":"[PATCH 04/16] reftable/block: simplify how we track restart points","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-31T08:41:22Z","receivedAt":"2025-03-31T08:41:26Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Restart points record the location of reftable records that do not use\nprefix compression and are used to perform a binary search inside of a\nblock. These restart points are encoded at the end of a block, between\nthe record data and the footer of a table.\n\nThe block structure contains three different variables related to these\nrestart points:\n\n  - The block length contains the length of the reftable block up to the\n    restart points.\n\n  - The restart count contains the number of restart points contained in\n    the block.\n\n  - The restart bytes variable tracks where the restart point data\n    begins.\n\nTracking all three of these variables is unnecessary though as the data\ncan be derived from one another: the block length without restart points\nis the exact same as the offset of the restart count data, which we\nalready track via the `restart_bytes` data.\n\nRefactor the code so that we track the location of restart bytes not as\na pointer, but instead as an offset. This allows us to trivially get rid\nof the `block_len` variable as described above. This avoids having the\nconfusing `block_len` variable and allows us to do less bookkeeping\noverall.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/block.c | 25 ++++++++++++-------------\n reftable/block.h |  8 +++++---\n reftable/table.c |  2 +-\n 3 files changed, 18 insertions(+), 17 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 97740187259..f2567a8f0fd 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -216,10 +216,9 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tuint32_t full_block_size = table_block_size;\n \tuint8_t typ = block->data[header_off];\n \tuint32_t sz = reftable_get_be24(block->data + header_off + 1);\n-\tint err = 0;\n-\tuint16_t restart_count = 0;\n-\tuint32_t restart_start = 0;\n-\tuint8_t *restart_bytes = NULL;\n+\tuint16_t restart_count;\n+\tuint32_t restart_off;\n+\tint err;\n \n \tblock_source_return_block(&br->block);\n \n@@ -300,8 +299,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t}\n \n \trestart_count = reftable_get_be16(block->data + sz - 2);\n-\trestart_start = sz - 2 - 3 * restart_count;\n-\trestart_bytes = block->data + restart_start;\n+\trestart_off = sz - 2 - 3 * restart_count;\n \n \t/* transfer ownership. */\n \tbr->block = *block;\n@@ -309,11 +307,12 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tblock->len = 0;\n \n \tbr->hash_size = hash_size;\n-\tbr->block_len = restart_start;\n+\tbr->restart_off = restart_off;\n \tbr->full_block_size = full_block_size;\n \tbr->header_off = header_off;\n \tbr->restart_count = restart_count;\n-\tbr->restart_bytes = restart_bytes;\n+\n+\terr = 0;\n \n done:\n \treturn err;\n@@ -337,7 +336,7 @@ int block_reader_first_key(const struct block_reader *br, struct reftable_buf *k\n \tint off = br->header_off + 4, n;\n \tstruct string_view in = {\n \t\t.buf = br->block.data + off,\n-\t\t.len = br->block_len - off,\n+\t\t.len = br->restart_off - off,\n \t};\n \tuint8_t extra = 0;\n \n@@ -354,13 +353,13 @@ int block_reader_first_key(const struct block_reader *br, struct reftable_buf *k\n \n static uint32_t block_reader_restart_offset(const struct block_reader *br, size_t idx)\n {\n-\treturn reftable_get_be24(br->restart_bytes + 3 * idx);\n+\treturn reftable_get_be24(br->block.data + br->restart_off + 3 * idx);\n }\n \n void block_iter_seek_start(struct block_iter *it, const struct block_reader *br)\n {\n \tit->block = br->block.data;\n-\tit->block_len = br->block_len;\n+\tit->block_len = br->restart_off;\n \tit->hash_size = br->hash_size;\n \treftable_buf_reset(&it->last_key);\n \tit->next_off = br->header_off + 4;\n@@ -378,7 +377,7 @@ static int restart_needle_less(size_t idx, void *_args)\n \tuint32_t off = block_reader_restart_offset(args->reader, idx);\n \tstruct string_view in = {\n \t\t.buf = args->reader->block.data + off,\n-\t\t.len = args->reader->block_len - off,\n+\t\t.len = args->reader->restart_off - off,\n \t};\n \tuint64_t prefix_len, suffix_len;\n \tuint8_t extra;\n@@ -505,7 +504,7 @@ int block_iter_seek_key(struct block_iter *it, const struct block_reader *br,\n \telse\n \t\tit->next_off = br->header_off + 4;\n \tit->block = br->block.data;\n-\tit->block_len = br->block_len;\n+\tit->block_len = br->restart_off;\n \tit->hash_size = br->hash_size;\n \n \terr = reftable_record_init(&rec, block_reader_type(br));\ndiff --git a/reftable/block.h b/reftable/block.h\nindex 203b07d9a44..b78f322e646 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -79,10 +79,12 @@ struct block_reader {\n \tunsigned char *uncompressed_data;\n \tsize_t uncompressed_cap;\n \n-\t/* size of the data, excluding restart data. */\n-\tuint32_t block_len;\n-\tuint8_t *restart_bytes;\n+\t/*\n+\t * Restart point data. Restart points are located after the block's\n+\t * record data.\n+\t */\n \tuint16_t restart_count;\n+\tuint32_t restart_off;\n \n \t/* size of the data in the file. For log blocks, this is the compressed\n \t * size. */\ndiff --git a/reftable/table.c b/reftable/table.c\nindex d18e17b0d44..ec84545707c 100644\n--- a/reftable/table.c\n+++ b/reftable/table.c\n@@ -838,7 +838,7 @@ int reftable_table_print_blocks(const char *tablename)\n \t\tprintf(\"%s:\\n\", sections[i].name);\n \n \t\twhile (1) {\n-\t\t\tprintf(\"  - length: %u\\n\", ti.br.block_len);\n+\t\t\tprintf(\"  - length: %u\\n\", ti.br.restart_off);\n \t\t\tprintf(\"    restarts: %u\\n\", ti.br.restart_count);\n \n \t\t\terr = table_iter_next_block(&ti);\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515342","messageId":"20250331-pks-reftable-polishing-v1-5-ebed5247434c@pks.im","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im","subject":"[PATCH 05/16] reftable/table: move reading block into block reader","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-31T08:41:23Z","receivedAt":"2025-03-31T08:41:27Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The logic to read blocks from a reftable is scattered across both the\ntable and the block subsystems. Besides causing somewhat fuzzy\nresponsibilities, it also means that we have to awkwardly pass around\nthe ownership of blocks between the subsystems.\n\nRefactor the code so that we stop passing the block when initializing a\nreader, but instead by passing in the block source plus the offset at\nwhich we we're supposed to read a block. Like this, the ownership of the\nblock itself doesn't need to get handed over as the block reader is the\none owning the block right from the start.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/block.c                | 87 ++++++++++++++++++++++++++---------------\n reftable/block.h                |  8 ++--\n reftable/table.c                | 65 +++---------------------------\n t/unit-tests/t-reftable-block.c | 76 ++++++++++++++++++-----------------\n 4 files changed, 107 insertions(+), 129 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex f2567a8f0fd..2517108b8ef 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -209,31 +209,57 @@ int block_writer_finish(struct block_writer *w)\n \treturn w->next;\n }\n \n-int block_reader_init(struct block_reader *br, struct reftable_block *block,\n-\t\t      uint32_t header_off, uint32_t table_block_size,\n-\t\t      uint32_t hash_size)\n+static int read_block(struct reftable_block_source *source,\n+\t\t      struct reftable_block *dest, uint64_t off,\n+\t\t      uint32_t sz)\n {\n+\tsize_t size = block_source_size(source);\n+\tblock_source_return_block(dest);\n+\tif (off >= size)\n+\t\treturn 0;\n+\tif (off + sz > size)\n+\t\tsz = size - off;\n+\treturn block_source_read_block(source, dest, off, sz);\n+}\n+\n+int block_reader_init(struct block_reader *br,\n+\t\t      struct reftable_block_source *source,\n+\t\t      uint32_t offset, uint32_t header_size,\n+\t\t      uint32_t table_block_size, uint32_t hash_size)\n+{\n+\tuint32_t guess_block_size = table_block_size ?\n+\t\ttable_block_size : DEFAULT_BLOCK_SIZE;\n \tuint32_t full_block_size = table_block_size;\n-\tuint8_t typ = block->data[header_off];\n-\tuint32_t sz = reftable_get_be24(block->data + header_off + 1);\n \tuint16_t restart_count;\n \tuint32_t restart_off;\n+\tuint32_t block_size;\n+\tuint8_t block_type;\n \tint err;\n \n-\tblock_source_return_block(&br->block);\n+\terr = read_block(source, &br->block, offset, guess_block_size);\n+\tif (err < 0)\n+\t\tgoto done;\n \n-\tif (!reftable_is_block_type(typ)) {\n-\t\terr =  REFTABLE_FORMAT_ERROR;\n+\tblock_type = br->block.data[header_size];\n+\tif (!reftable_is_block_type(block_type)) {\n+\t\terr = REFTABLE_FORMAT_ERROR;\n \t\tgoto done;\n \t}\n \n-\tif (typ == BLOCK_TYPE_LOG) {\n-\t\tuint32_t block_header_skip = 4 + header_off;\n-\t\tuLong dst_len = sz - block_header_skip;\n-\t\tuLong src_len = block->len - block_header_skip;\n+\tblock_size = reftable_get_be24(br->block.data + header_size + 1);\n+\tif (block_size > guess_block_size) {\n+\t\terr = read_block(source, &br->block, offset, block_size);\n+\t\tif (err < 0)\n+\t\t\tgoto done;\n+\t}\n+\n+\tif (block_type == BLOCK_TYPE_LOG) {\n+\t\tuint32_t block_header_skip = 4 + header_size;\n+\t\tuLong dst_len = block_size - block_header_skip;\n+\t\tuLong src_len = br->block.len - block_header_skip;\n \n \t\t/* Log blocks specify the *uncompressed* size in their header. */\n-\t\tREFTABLE_ALLOC_GROW_OR_NULL(br->uncompressed_data, sz,\n+\t\tREFTABLE_ALLOC_GROW_OR_NULL(br->uncompressed_data, block_size,\n \t\t\t\t\t    br->uncompressed_cap);\n \t\tif (!br->uncompressed_data) {\n \t\t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n@@ -241,7 +267,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\t}\n \n \t\t/* Copy over the block header verbatim. It's not compressed. */\n-\t\tmemcpy(br->uncompressed_data, block->data, block_header_skip);\n+\t\tmemcpy(br->uncompressed_data, br->block.data, block_header_skip);\n \n \t\tif (!br->zstream) {\n \t\t\tREFTABLE_CALLOC_ARRAY(br->zstream, 1);\n@@ -259,7 +285,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\t\tgoto done;\n \t\t}\n \n-\t\tbr->zstream->next_in = block->data + block_header_skip;\n+\t\tbr->zstream->next_in = br->block.data + block_header_skip;\n \t\tbr->zstream->avail_in = src_len;\n \t\tbr->zstream->next_out = br->uncompressed_data + block_header_skip;\n \t\tbr->zstream->avail_out = dst_len;\n@@ -278,43 +304,41 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\t}\n \t\terr = 0;\n \n-\t\tif (br->zstream->total_out + block_header_skip != sz) {\n+\t\tif (br->zstream->total_out + block_header_skip != block_size) {\n \t\t\terr = REFTABLE_FORMAT_ERROR;\n \t\t\tgoto done;\n \t\t}\n \n \t\t/* We're done with the input data. */\n-\t\tblock_source_return_block(block);\n-\t\tblock->data = br->uncompressed_data;\n-\t\tblock->len = sz;\n+\t\tblock_source_return_block(&br->block);\n+\t\tbr->block.data = br->uncompressed_data;\n+\t\tbr->block.len = block_size;\n \t\tfull_block_size = src_len + block_header_skip - br->zstream->avail_in;\n \t} else if (full_block_size == 0) {\n-\t\tfull_block_size = sz;\n-\t} else if (sz < full_block_size && sz < block->len &&\n-\t\t   block->data[sz] != 0) {\n+\t\tfull_block_size = block_size;\n+\t} else if (block_size < full_block_size && block_size < br->block.len &&\n+\t\t   br->block.data[block_size] != 0) {\n \t\t/* If the block is smaller than the full block size, it is\n \t\t   padded (data followed by '\\0') or the next block is\n \t\t   unaligned. */\n-\t\tfull_block_size = sz;\n+\t\tfull_block_size = block_size;\n \t}\n \n-\trestart_count = reftable_get_be16(block->data + sz - 2);\n-\trestart_off = sz - 2 - 3 * restart_count;\n-\n-\t/* transfer ownership. */\n-\tbr->block = *block;\n-\tblock->data = NULL;\n-\tblock->len = 0;\n+\trestart_count = reftable_get_be16(br->block.data + block_size - 2);\n+\trestart_off = block_size - 2 - 3 * restart_count;\n \n+\tbr->block_type = block_type;\n \tbr->hash_size = hash_size;\n \tbr->restart_off = restart_off;\n \tbr->full_block_size = full_block_size;\n-\tbr->header_off = header_off;\n+\tbr->header_off = header_size;\n \tbr->restart_count = restart_count;\n \n \terr = 0;\n \n done:\n+\tif (err < 0)\n+\t\tblock_reader_release(br);\n \treturn err;\n }\n \n@@ -324,6 +348,7 @@ void block_reader_release(struct block_reader *br)\n \treftable_free(br->zstream);\n \treftable_free(br->uncompressed_data);\n \tblock_source_return_block(&br->block);\n+\tmemset(br, 0, sizeof(*br));\n }\n \n uint8_t block_reader_type(const struct block_reader *r)\ndiff --git a/reftable/block.h b/reftable/block.h\nindex b78f322e646..6afb1b2952a 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -89,12 +89,14 @@ struct block_reader {\n \t/* size of the data in the file. For log blocks, this is the compressed\n \t * size. */\n \tuint32_t full_block_size;\n+\tuint8_t block_type;\n };\n \n /* initializes a block reader. */\n-int block_reader_init(struct block_reader *br, struct reftable_block *bl,\n-\t\t      uint32_t header_off, uint32_t table_block_size,\n-\t\t      uint32_t hash_size);\n+int block_reader_init(struct block_reader *br,\n+\t\t      struct reftable_block_source *source,\n+\t\t      uint32_t offset, uint32_t header_size,\n+\t\t      uint32_t table_block_size, uint32_t hash_size);\n \n void block_reader_release(struct block_reader *br);\n \ndiff --git a/reftable/table.c b/reftable/table.c\nindex ec84545707c..7c0f1c9e6e3 100644\n--- a/reftable/table.c\n+++ b/reftable/table.c\n@@ -30,23 +30,6 @@ table_offsets_for(struct reftable_table *t, uint8_t typ)\n \tabort();\n }\n \n-static int table_get_block(struct reftable_table *t,\n-\t\t\t   struct reftable_block *dest, uint64_t off,\n-\t\t\t   uint32_t sz)\n-{\n-\tssize_t bytes_read;\n-\tif (off >= t->size)\n-\t\treturn 0;\n-\tif (off + sz > t->size)\n-\t\tsz = t->size - off;\n-\n-\tbytes_read = block_source_read_block(&t->source, dest, off, sz);\n-\tif (bytes_read < 0)\n-\t\treturn (int)bytes_read;\n-\n-\treturn 0;\n-}\n-\n enum reftable_hash reftable_table_hash_id(struct reftable_table *t)\n {\n \treturn t->hash_id;\n@@ -180,64 +163,28 @@ static void table_iter_block_done(struct table_iter *ti)\n \tblock_iter_reset(&ti->bi);\n }\n \n-static int32_t extract_block_size(uint8_t *data, uint8_t *typ, uint64_t off,\n-\t\t\t\t  int version)\n-{\n-\tint32_t result = 0;\n-\n-\tif (off == 0) {\n-\t\tdata += header_size(version);\n-\t}\n-\n-\t*typ = data[0];\n-\tif (reftable_is_block_type(*typ)) {\n-\t\tresult = reftable_get_be24(data + 1);\n-\t}\n-\treturn result;\n-}\n-\n int table_init_block_reader(struct reftable_table *t, struct block_reader *br,\n \t\t\t    uint64_t next_off, uint8_t want_typ)\n {\n-\tint32_t guess_block_size = t->block_size ? t->block_size :\n-\t\t\t\t\t\t\t DEFAULT_BLOCK_SIZE;\n-\tstruct reftable_block block = { NULL };\n-\tuint8_t block_typ = 0;\n-\tint err = 0;\n \tuint32_t header_off = next_off ? 0 : header_size(t->version);\n-\tint32_t block_size = 0;\n+\tint err;\n \n \tif (next_off >= t->size)\n \t\treturn 1;\n \n-\terr = table_get_block(t, &block, next_off, guess_block_size);\n+\terr = block_reader_init(br, &t->source, next_off, header_off,\n+\t\t\t\tt->block_size, hash_size(t->hash_id));\n \tif (err < 0)\n \t\tgoto done;\n \n-\tblock_size = extract_block_size(block.data, &block_typ, next_off,\n-\t\t\t\t\tt->version);\n-\tif (block_size < 0) {\n-\t\terr = block_size;\n-\t\tgoto done;\n-\t}\n-\tif (want_typ != BLOCK_TYPE_ANY && block_typ != want_typ) {\n+\tif (want_typ != BLOCK_TYPE_ANY && br->block_type != want_typ) {\n \t\terr = 1;\n \t\tgoto done;\n \t}\n \n-\tif (block_size > guess_block_size) {\n-\t\tblock_source_return_block(&block);\n-\t\terr = table_get_block(t, &block, next_off, block_size);\n-\t\tif (err < 0) {\n-\t\t\tgoto done;\n-\t\t}\n-\t}\n-\n-\terr = block_reader_init(br, &block, header_off, t->block_size,\n-\t\t\t\thash_size(t->hash_id));\n done:\n-\tblock_source_return_block(&block);\n-\n+\tif (err)\n+\t\tblock_reader_release(br);\n \treturn err;\n }\n \ndiff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c\nindex 8bb40482347..dbec6a5d0d0 100644\n--- a/t/unit-tests/t-reftable-block.c\n+++ b/t/unit-tests/t-reftable-block.c\n@@ -19,7 +19,7 @@ static void t_ref_block_read_write(void)\n \tstruct reftable_record recs[30];\n \tconst size_t N = ARRAY_SIZE(recs);\n \tconst size_t block_size = 1024;\n-\tstruct reftable_block block = { 0 };\n+\tstruct reftable_block_source source = { 0 };\n \tstruct block_writer bw = {\n \t\t.last_key = REFTABLE_BUF_INIT,\n \t};\n@@ -30,13 +30,14 @@ static void t_ref_block_read_write(void)\n \tint ret;\n \tstruct block_reader br = { 0 };\n \tstruct block_iter it = BLOCK_ITER_INIT;\n-\tstruct reftable_buf want = REFTABLE_BUF_INIT, buf = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf want = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf block = REFTABLE_BUF_INIT;\n \n-\tREFTABLE_CALLOC_ARRAY(block.data, block_size);\n-\tcheck(block.data != NULL);\n+\tREFTABLE_CALLOC_ARRAY(block.buf, block_size);\n+\tcheck(block.buf != NULL);\n \tblock.len = block_size;\n-\tblock_source_from_buf(&block.source ,&buf);\n-\tret = block_writer_init(&bw, BLOCK_TYPE_REF, block.data, block_size,\n+\n+\tret = block_writer_init(&bw, BLOCK_TYPE_REF, (uint8_t *) block.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -62,7 +63,8 @@ static void t_ref_block_read_write(void)\n \n \tblock_writer_release(&bw);\n \n-\tblock_reader_init(&br, &block, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n+\tblock_source_from_buf(&source ,&block);\n+\tblock_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n \tblock_iter_seek_start(&it, &br);\n \n@@ -100,9 +102,8 @@ static void t_ref_block_read_write(void)\n \tblock_reader_release(&br);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n-\tblock_source_return_block(&br.block);\n \treftable_buf_release(&want);\n-\treftable_buf_release(&buf);\n+\treftable_buf_release(&block);\n \tfor (i = 0; i < N; i++)\n \t\treftable_record_release(&recs[i]);\n }\n@@ -113,7 +114,7 @@ static void t_log_block_read_write(void)\n \tstruct reftable_record recs[30];\n \tconst size_t N = ARRAY_SIZE(recs);\n \tconst size_t block_size = 2048;\n-\tstruct reftable_block block = { 0 };\n+\tstruct reftable_block_source source = { 0 };\n \tstruct block_writer bw = {\n \t\t.last_key = REFTABLE_BUF_INIT,\n \t};\n@@ -124,13 +125,14 @@ static void t_log_block_read_write(void)\n \tint ret;\n \tstruct block_reader br = { 0 };\n \tstruct block_iter it = BLOCK_ITER_INIT;\n-\tstruct reftable_buf want = REFTABLE_BUF_INIT, buf = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf want = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf block = REFTABLE_BUF_INIT;\n \n-\tREFTABLE_CALLOC_ARRAY(block.data, block_size);\n-\tcheck(block.data != NULL);\n+\tREFTABLE_CALLOC_ARRAY(block.buf, block_size);\n+\tcheck(block.buf != NULL);\n \tblock.len = block_size;\n-\tblock_source_from_buf(&block.source ,&buf);\n-\tret = block_writer_init(&bw, BLOCK_TYPE_LOG, block.data, block_size,\n+\n+\tret = block_writer_init(&bw, BLOCK_TYPE_LOG, (uint8_t *) block.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -151,7 +153,8 @@ static void t_log_block_read_write(void)\n \n \tblock_writer_release(&bw);\n \n-\tblock_reader_init(&br, &block, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n+\tblock_source_from_buf(&source, &block);\n+\tblock_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n \tblock_iter_seek_start(&it, &br);\n \n@@ -190,9 +193,8 @@ static void t_log_block_read_write(void)\n \tblock_reader_release(&br);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n-\tblock_source_return_block(&br.block);\n \treftable_buf_release(&want);\n-\treftable_buf_release(&buf);\n+\treftable_buf_release(&block);\n \tfor (i = 0; i < N; i++)\n \t\treftable_record_release(&recs[i]);\n }\n@@ -203,7 +205,7 @@ static void t_obj_block_read_write(void)\n \tstruct reftable_record recs[30];\n \tconst size_t N = ARRAY_SIZE(recs);\n \tconst size_t block_size = 1024;\n-\tstruct reftable_block block = { 0 };\n+\tstruct reftable_block_source source = { 0 };\n \tstruct block_writer bw = {\n \t\t.last_key = REFTABLE_BUF_INIT,\n \t};\n@@ -214,13 +216,14 @@ static void t_obj_block_read_write(void)\n \tint ret;\n \tstruct block_reader br = { 0 };\n \tstruct block_iter it = BLOCK_ITER_INIT;\n-\tstruct reftable_buf want = REFTABLE_BUF_INIT, buf = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf want = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf block = REFTABLE_BUF_INIT;\n \n-\tREFTABLE_CALLOC_ARRAY(block.data, block_size);\n-\tcheck(block.data != NULL);\n+\tREFTABLE_CALLOC_ARRAY(block.buf, block_size);\n+\tcheck(block.buf != NULL);\n \tblock.len = block_size;\n-\tblock_source_from_buf(&block.source, &buf);\n-\tret = block_writer_init(&bw, BLOCK_TYPE_OBJ, block.data, block_size,\n+\n+\tret = block_writer_init(&bw, BLOCK_TYPE_OBJ, (uint8_t *) block.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -243,7 +246,8 @@ static void t_obj_block_read_write(void)\n \n \tblock_writer_release(&bw);\n \n-\tblock_reader_init(&br, &block, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n+\tblock_source_from_buf(&source, &block);\n+\tblock_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n \tblock_iter_seek_start(&it, &br);\n \n@@ -273,9 +277,8 @@ static void t_obj_block_read_write(void)\n \tblock_reader_release(&br);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n-\tblock_source_return_block(&br.block);\n \treftable_buf_release(&want);\n-\treftable_buf_release(&buf);\n+\treftable_buf_release(&block);\n \tfor (i = 0; i < N; i++)\n \t\treftable_record_release(&recs[i]);\n }\n@@ -286,7 +289,7 @@ static void t_index_block_read_write(void)\n \tstruct reftable_record recs[30];\n \tconst size_t N = ARRAY_SIZE(recs);\n \tconst size_t block_size = 1024;\n-\tstruct reftable_block block = { 0 };\n+\tstruct reftable_block_source source = { 0 };\n \tstruct block_writer bw = {\n \t\t.last_key = REFTABLE_BUF_INIT,\n \t};\n@@ -298,13 +301,14 @@ static void t_index_block_read_write(void)\n \tint ret;\n \tstruct block_reader br = { 0 };\n \tstruct block_iter it = BLOCK_ITER_INIT;\n-\tstruct reftable_buf want = REFTABLE_BUF_INIT, buf = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf want = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf block = REFTABLE_BUF_INIT;\n \n-\tREFTABLE_CALLOC_ARRAY(block.data, block_size);\n-\tcheck(block.data != NULL);\n+\tREFTABLE_CALLOC_ARRAY(block.buf, block_size);\n+\tcheck(block.buf != NULL);\n \tblock.len = block_size;\n-\tblock_source_from_buf(&block.source, &buf);\n-\tret = block_writer_init(&bw, BLOCK_TYPE_INDEX, block.data, block_size,\n+\n+\tret = block_writer_init(&bw, BLOCK_TYPE_INDEX, (uint8_t *) block.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -327,7 +331,8 @@ static void t_index_block_read_write(void)\n \n \tblock_writer_release(&bw);\n \n-\tblock_reader_init(&br, &block, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n+\tblock_source_from_buf(&source, &block);\n+\tblock_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n \tblock_iter_seek_start(&it, &br);\n \n@@ -365,9 +370,8 @@ static void t_index_block_read_write(void)\n \tblock_reader_release(&br);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n-\tblock_source_return_block(&br.block);\n \treftable_buf_release(&want);\n-\treftable_buf_release(&buf);\n+\treftable_buf_release(&block);\n \tfor (i = 0; i < N; i++)\n \t\treftable_record_release(&recs[i]);\n }\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515344","messageId":"20250331-pks-reftable-polishing-v1-6-ebed5247434c@pks.im","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im","subject":"[PATCH 06/16] reftable/block: rename `block` to `block_data`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-31T08:41:24Z","receivedAt":"2025-03-31T08:41:28Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The `reftable_block` structure associates a byte slice with a block\nsource. As such it only holds the data of a reftable block without\nactually encoding any of the details for how to access that data.\n\nRename the structure to instead be called `reftable_block_data`. Besides\nclarifying that this really only holds data, it also allows us to rename\nthe `reftable_block_reader` to `reftable_block` in the next commit, as\nthis is the structure that actually encapsulates access to the reftable\nblocks.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/block.c                    | 10 ++++-----\n reftable/block.h                    |  2 +-\n reftable/blocksource.c              | 44 ++++++++++++++++++-------------------\n reftable/blocksource.h              | 10 ++++-----\n reftable/iter.c                     |  4 ++--\n reftable/reftable-blocksource.h     | 14 ++++++------\n reftable/table.c                    | 14 ++++++------\n t/unit-tests/t-reftable-readwrite.c | 10 ++++-----\n 8 files changed, 54 insertions(+), 54 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 2517108b8ef..ad162ecdbf6 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -210,16 +210,16 @@ int block_writer_finish(struct block_writer *w)\n }\n \n static int read_block(struct reftable_block_source *source,\n-\t\t      struct reftable_block *dest, uint64_t off,\n+\t\t      struct reftable_block_data *dest, uint64_t off,\n \t\t      uint32_t sz)\n {\n \tsize_t size = block_source_size(source);\n-\tblock_source_return_block(dest);\n+\tblock_source_release_data(dest);\n \tif (off >= size)\n \t\treturn 0;\n \tif (off + sz > size)\n \t\tsz = size - off;\n-\treturn block_source_read_block(source, dest, off, sz);\n+\treturn block_source_read_data(source, dest, off, sz);\n }\n \n int block_reader_init(struct block_reader *br,\n@@ -310,7 +310,7 @@ int block_reader_init(struct block_reader *br,\n \t\t}\n \n \t\t/* We're done with the input data. */\n-\t\tblock_source_return_block(&br->block);\n+\t\tblock_source_release_data(&br->block);\n \t\tbr->block.data = br->uncompressed_data;\n \t\tbr->block.len = block_size;\n \t\tfull_block_size = src_len + block_header_skip - br->zstream->avail_in;\n@@ -347,7 +347,7 @@ void block_reader_release(struct block_reader *br)\n \tinflateEnd(br->zstream);\n \treftable_free(br->zstream);\n \treftable_free(br->uncompressed_data);\n-\tblock_source_return_block(&br->block);\n+\tblock_source_release_data(&br->block);\n \tmemset(br, 0, sizeof(*br));\n }\n \ndiff --git a/reftable/block.h b/reftable/block.h\nindex 6afb1b2952a..c55d47c3c52 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -71,7 +71,7 @@ struct block_reader {\n \tuint32_t header_off;\n \n \t/* the memory block */\n-\tstruct reftable_block block;\n+\tstruct reftable_block_data block;\n \tuint32_t hash_size;\n \n \t/* Uncompressed data for log entries. */\ndiff --git a/reftable/blocksource.c b/reftable/blocksource.c\nindex bc785506fb1..573c81287fe 100644\n--- a/reftable/blocksource.c\n+++ b/reftable/blocksource.c\n@@ -13,15 +13,15 @@\n #include \"reftable-blocksource.h\"\n #include \"reftable-error.h\"\n \n-void block_source_return_block(struct reftable_block *block)\n+void block_source_release_data(struct reftable_block_data *data)\n {\n-\tstruct reftable_block_source source = block->source;\n-\tif (block && source.ops)\n-\t\tsource.ops->return_block(source.arg, block);\n-\tblock->data = NULL;\n-\tblock->len = 0;\n-\tblock->source.ops = NULL;\n-\tblock->source.arg = NULL;\n+\tstruct reftable_block_source source = data->source;\n+\tif (data && source.ops)\n+\t\tsource.ops->release_data(source.arg, data);\n+\tdata->data = NULL;\n+\tdata->len = 0;\n+\tdata->source.ops = NULL;\n+\tdata->source.arg = NULL;\n }\n \n void block_source_close(struct reftable_block_source *source)\n@@ -34,11 +34,11 @@ void block_source_close(struct reftable_block_source *source)\n \tsource->ops = NULL;\n }\n \n-ssize_t block_source_read_block(struct reftable_block_source *source,\n-\t\t\t\tstruct reftable_block *dest, uint64_t off,\n-\t\t\t\tuint32_t size)\n+ssize_t block_source_read_data(struct reftable_block_source *source,\n+\t\t\t       struct reftable_block_data *dest, uint64_t off,\n+\t\t\t       uint32_t size)\n {\n-\tssize_t result = source->ops->read_block(source->arg, dest, off, size);\n+\tssize_t result = source->ops->read_data(source->arg, dest, off, size);\n \tdest->source = *source;\n \treturn result;\n }\n@@ -48,7 +48,7 @@ uint64_t block_source_size(struct reftable_block_source *source)\n \treturn source->ops->size(source->arg);\n }\n \n-static void reftable_buf_return_block(void *b REFTABLE_UNUSED, struct reftable_block *dest)\n+static void reftable_buf_release_data(void *b REFTABLE_UNUSED, struct reftable_block_data *dest)\n {\n \tif (dest->len)\n \t\tmemset(dest->data, 0xff, dest->len);\n@@ -59,8 +59,8 @@ static void reftable_buf_close(void *b REFTABLE_UNUSED)\n {\n }\n \n-static ssize_t reftable_buf_read_block(void *v, struct reftable_block *dest,\n-\t\t\t\t       uint64_t off, uint32_t size)\n+static ssize_t reftable_buf_read_data(void *v, struct reftable_block_data *dest,\n+\t\t\t\t      uint64_t off, uint32_t size)\n {\n \tstruct reftable_buf *b = v;\n \tassert(off + size <= b->len);\n@@ -79,8 +79,8 @@ static uint64_t reftable_buf_size(void *b)\n \n static struct reftable_block_source_vtable reftable_buf_vtable = {\n \t.size = &reftable_buf_size,\n-\t.read_block = &reftable_buf_read_block,\n-\t.return_block = &reftable_buf_return_block,\n+\t.read_data = &reftable_buf_read_data,\n+\t.release_data = &reftable_buf_release_data,\n \t.close = &reftable_buf_close,\n };\n \n@@ -102,7 +102,7 @@ static uint64_t file_size(void *b)\n \treturn ((struct file_block_source *)b)->size;\n }\n \n-static void file_return_block(void *b REFTABLE_UNUSED, struct reftable_block *dest REFTABLE_UNUSED)\n+static void file_release_data(void *b REFTABLE_UNUSED, struct reftable_block_data *dest REFTABLE_UNUSED)\n {\n }\n \n@@ -113,8 +113,8 @@ static void file_close(void *v)\n \treftable_free(b);\n }\n \n-static ssize_t file_read_block(void *v, struct reftable_block *dest, uint64_t off,\n-\t\t\t       uint32_t size)\n+static ssize_t file_read_data(void *v, struct reftable_block_data *dest, uint64_t off,\n+\t\t\t      uint32_t size)\n {\n \tstruct file_block_source *b = v;\n \tassert(off + size <= b->size);\n@@ -125,8 +125,8 @@ static ssize_t file_read_block(void *v, struct reftable_block *dest, uint64_t of\n \n static struct reftable_block_source_vtable file_vtable = {\n \t.size = &file_size,\n-\t.read_block = &file_read_block,\n-\t.return_block = &file_return_block,\n+\t.read_data = &file_read_data,\n+\t.release_data = &file_release_data,\n \t.close = &file_close,\n };\n \ndiff --git a/reftable/blocksource.h b/reftable/blocksource.h\nindex 639b9a1a3c5..a110e059580 100644\n--- a/reftable/blocksource.h\n+++ b/reftable/blocksource.h\n@@ -12,7 +12,7 @@\n #include \"system.h\"\n \n struct reftable_block_source;\n-struct reftable_block;\n+struct reftable_block_data;\n struct reftable_buf;\n \n /*\n@@ -24,9 +24,9 @@ void block_source_close(struct reftable_block_source *source);\n /*\n  * Read a block of length `size` from the source at the given `off`.\n  */\n-ssize_t block_source_read_block(struct reftable_block_source *source,\n-\t\t\t\tstruct reftable_block *dest, uint64_t off,\n-\t\t\t\tuint32_t size);\n+ssize_t block_source_read_data(struct reftable_block_source *source,\n+\t\t\t       struct reftable_block_data *dest, uint64_t off,\n+\t\t\t       uint32_t size);\n \n /*\n  * Return the total length of the underlying resource.\n@@ -37,7 +37,7 @@ uint64_t block_source_size(struct reftable_block_source *source);\n  * Return a block to its original source, releasing any resources associated\n  * with it.\n  */\n-void block_source_return_block(struct reftable_block *block);\n+void block_source_release_data(struct reftable_block_data *data);\n \n /* Create an in-memory block source for reading reftables. */\n void block_source_from_buf(struct reftable_block_source *bs,\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex 6af6eb49396..37b354a6fb9 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -114,7 +114,7 @@ static void indexed_table_ref_iter_close(void *p)\n {\n \tstruct indexed_table_ref_iter *it = p;\n \tblock_iter_close(&it->cur);\n-\tblock_source_return_block(&it->block_reader.block);\n+\tblock_source_release_data(&it->block_reader.block);\n \treftable_free(it->offsets);\n \treftable_buf_release(&it->oid);\n }\n@@ -128,7 +128,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n \t\treturn 1;\n \t}\n \n-\tblock_source_return_block(&it->block_reader.block);\n+\tblock_source_release_data(&it->block_reader.block);\n \n \toff = it->offsets[it->offset_idx++];\n \terr = table_init_block_reader(it->table, &it->block_reader, off,\ndiff --git a/reftable/reftable-blocksource.h b/reftable/reftable-blocksource.h\nindex 96430b629e4..f5ba867bd60 100644\n--- a/reftable/reftable-blocksource.h\n+++ b/reftable/reftable-blocksource.h\n@@ -21,7 +21,7 @@ struct reftable_block_source {\n \n /* a contiguous segment of bytes. It keeps track of its generating block_source\n  * so it can return itself into the pool. */\n-struct reftable_block {\n+struct reftable_block_data {\n \tuint8_t *data;\n \tsize_t len;\n \tstruct reftable_block_source source;\n@@ -29,20 +29,20 @@ struct reftable_block {\n \n /* block_source_vtable are the operations that make up block_source */\n struct reftable_block_source_vtable {\n-\t/* returns the size of a block source */\n+\t/* Returns the size of a block source. */\n \tuint64_t (*size)(void *source);\n \n \t/*\n \t * Reads a segment from the block source. It is an error to read beyond\n \t * the end of the block.\n \t */\n-\tssize_t (*read_block)(void *source, struct reftable_block *dest,\n-\t\t\t     uint64_t off, uint32_t size);\n+\tssize_t (*read_data)(void *source, struct reftable_block_data *dest,\n+\t\t\tuint64_t off, uint32_t size);\n \n-\t/* mark the block as read; may return the data back to malloc */\n-\tvoid (*return_block)(void *source, struct reftable_block *blockp);\n+\t/* Mark the block as read; may release the data. */\n+\tvoid (*release_data)(void *source, struct reftable_block_data *data);\n \n-\t/* release all resources associated with the block source */\n+\t/* Release all resources associated with the block source. */\n \tvoid (*close)(void *source);\n };\n \ndiff --git a/reftable/table.c b/reftable/table.c\nindex 7c0f1c9e6e3..a5ce96fa51c 100644\n--- a/reftable/table.c\n+++ b/reftable/table.c\n@@ -526,8 +526,8 @@ int reftable_table_init_log_iterator(struct reftable_table *t,\n int reftable_table_new(struct reftable_table **out,\n \t\t       struct reftable_block_source *source, char const *name)\n {\n-\tstruct reftable_block footer = { 0 };\n-\tstruct reftable_block header = { 0 };\n+\tstruct reftable_block_data footer = { 0 };\n+\tstruct reftable_block_data header = { 0 };\n \tstruct reftable_table *t;\n \tuint64_t file_size = block_source_size(source);\n \tuint32_t read_size;\n@@ -550,7 +550,7 @@ int reftable_table_new(struct reftable_table **out,\n \t\tgoto done;\n \t}\n \n-\tbytes_read = block_source_read_block(source, &header, 0, read_size);\n+\tbytes_read = block_source_read_data(source, &header, 0, read_size);\n \tif (bytes_read < 0 || (size_t)bytes_read != read_size) {\n \t\terr = REFTABLE_IO_ERROR;\n \t\tgoto done;\n@@ -576,8 +576,8 @@ int reftable_table_new(struct reftable_table **out,\n \tt->hash_id = 0;\n \tt->refcount = 1;\n \n-\tbytes_read = block_source_read_block(source, &footer, t->size,\n-\t\t\t\t\t     footer_size(t->version));\n+\tbytes_read = block_source_read_data(source, &footer, t->size,\n+\t\t\t\t\t    footer_size(t->version));\n \tif (bytes_read < 0 || (size_t)bytes_read != footer_size(t->version)) {\n \t\terr = REFTABLE_IO_ERROR;\n \t\tgoto done;\n@@ -590,8 +590,8 @@ int reftable_table_new(struct reftable_table **out,\n \t*out = t;\n \n done:\n-\tblock_source_return_block(&footer);\n-\tblock_source_return_block(&header);\n+\tblock_source_release_data(&footer);\n+\tblock_source_release_data(&header);\n \tif (err) {\n \t\tif (t)\n \t\t\treftable_free(t->name);\ndiff --git a/t/unit-tests/t-reftable-readwrite.c b/t/unit-tests/t-reftable-readwrite.c\nindex 3fba888cdaa..4c49129439e 100644\n--- a/t/unit-tests/t-reftable-readwrite.c\n+++ b/t/unit-tests/t-reftable-readwrite.c\n@@ -23,22 +23,22 @@ static void t_buffer(void)\n {\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tstruct reftable_block_source source = { 0 };\n-\tstruct reftable_block out = { 0 };\n+\tstruct reftable_block_data out = { 0 };\n \tint n;\n \tuint8_t in[] = \"hello\";\n \tcheck(!reftable_buf_add(&buf, in, sizeof(in)));\n \tblock_source_from_buf(&source, &buf);\n \tcheck_int(block_source_size(&source), ==, 6);\n-\tn = block_source_read_block(&source, &out, 0, sizeof(in));\n+\tn = block_source_read_data(&source, &out, 0, sizeof(in));\n \tcheck_int(n, ==, sizeof(in));\n \tcheck(!memcmp(in, out.data, n));\n-\tblock_source_return_block(&out);\n+\tblock_source_release_data(&out);\n \n-\tn = block_source_read_block(&source, &out, 1, 2);\n+\tn = block_source_read_data(&source, &out, 1, 2);\n \tcheck_int(n, ==, 2);\n \tcheck(!memcmp(out.data, \"el\", 2));\n \n-\tblock_source_return_block(&out);\n+\tblock_source_release_data(&out);\n \tblock_source_close(&source);\n \treftable_buf_release(&buf);\n }\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515346","messageId":"20250331-pks-reftable-polishing-v1-7-ebed5247434c@pks.im","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im","subject":"[PATCH 07/16] reftable/block: rename `block_reader` to `reftable_block`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-31T08:41:25Z","receivedAt":"2025-03-31T08:41:29Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The `block_reader` structure is used to access parsed data of a reftable\nblock. The structure is currently treated as an internal implementation\ndetail and not exposed via our public interfaces. The functionality\nprovided by the structure is useful to external users of the reftable\nlibrary though, for example when implementing consistency checks that\nneed to scan through the blocks manually.\n\nRename the structure to `reftable_block` now that the name has been made\navailable in the preceding commit. This name is in line with the naming\nschema used for other data structures like `reftable_table` in that it\ndescribes the underlying entity that it provides access to.\n\nThe new data structure isn't yet exposed via the public interface, which\nis left for a subsequent commit.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/block.c                | 142 ++++++++++++++++++++--------------------\n reftable/block.h                |  29 ++++----\n reftable/iter.c                 |   9 ++-\n reftable/iter.h                 |   2 +-\n reftable/table.c                |  46 ++++++-------\n reftable/table.h                |   8 ++-\n t/unit-tests/t-reftable-block.c | 102 ++++++++++++++---------------\n 7 files changed, 172 insertions(+), 166 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex ad162ecdbf6..d188665388d 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -222,10 +222,10 @@ static int read_block(struct reftable_block_source *source,\n \treturn block_source_read_data(source, dest, off, sz);\n }\n \n-int block_reader_init(struct block_reader *br,\n-\t\t      struct reftable_block_source *source,\n-\t\t      uint32_t offset, uint32_t header_size,\n-\t\t      uint32_t table_block_size, uint32_t hash_size)\n+int reftable_block_init(struct reftable_block *block,\n+\t\t\tstruct reftable_block_source *source,\n+\t\t\tuint32_t offset, uint32_t header_size,\n+\t\t\tuint32_t table_block_size, uint32_t hash_size)\n {\n \tuint32_t guess_block_size = table_block_size ?\n \t\ttable_block_size : DEFAULT_BLOCK_SIZE;\n@@ -236,19 +236,19 @@ int block_reader_init(struct block_reader *br,\n \tuint8_t block_type;\n \tint err;\n \n-\terr = read_block(source, &br->block, offset, guess_block_size);\n+\terr = read_block(source, &block->block, offset, guess_block_size);\n \tif (err < 0)\n \t\tgoto done;\n \n-\tblock_type = br->block.data[header_size];\n+\tblock_type = block->block.data[header_size];\n \tif (!reftable_is_block_type(block_type)) {\n \t\terr = REFTABLE_FORMAT_ERROR;\n \t\tgoto done;\n \t}\n \n-\tblock_size = reftable_get_be24(br->block.data + header_size + 1);\n+\tblock_size = reftable_get_be24(block->block.data + header_size + 1);\n \tif (block_size > guess_block_size) {\n-\t\terr = read_block(source, &br->block, offset, block_size);\n+\t\terr = read_block(source, &block->block, offset, block_size);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \t}\n@@ -256,39 +256,39 @@ int block_reader_init(struct block_reader *br,\n \tif (block_type == BLOCK_TYPE_LOG) {\n \t\tuint32_t block_header_skip = 4 + header_size;\n \t\tuLong dst_len = block_size - block_header_skip;\n-\t\tuLong src_len = br->block.len - block_header_skip;\n+\t\tuLong src_len = block->block.len - block_header_skip;\n \n \t\t/* Log blocks specify the *uncompressed* size in their header. */\n-\t\tREFTABLE_ALLOC_GROW_OR_NULL(br->uncompressed_data, block_size,\n-\t\t\t\t\t    br->uncompressed_cap);\n-\t\tif (!br->uncompressed_data) {\n+\t\tREFTABLE_ALLOC_GROW_OR_NULL(block->uncompressed_data, block_size,\n+\t\t\t\t\t    block->uncompressed_cap);\n+\t\tif (!block->uncompressed_data) {\n \t\t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\t\tgoto done;\n \t\t}\n \n \t\t/* Copy over the block header verbatim. It's not compressed. */\n-\t\tmemcpy(br->uncompressed_data, br->block.data, block_header_skip);\n+\t\tmemcpy(block->uncompressed_data, block->block.data, block_header_skip);\n \n-\t\tif (!br->zstream) {\n-\t\t\tREFTABLE_CALLOC_ARRAY(br->zstream, 1);\n-\t\t\tif (!br->zstream) {\n+\t\tif (!block->zstream) {\n+\t\t\tREFTABLE_CALLOC_ARRAY(block->zstream, 1);\n+\t\t\tif (!block->zstream) {\n \t\t\t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\t\t\tgoto done;\n \t\t\t}\n \n-\t\t\terr = inflateInit(br->zstream);\n+\t\t\terr = inflateInit(block->zstream);\n \t\t} else {\n-\t\t\terr = inflateReset(br->zstream);\n+\t\t\terr = inflateReset(block->zstream);\n \t\t}\n \t\tif (err != Z_OK) {\n \t\t\terr = REFTABLE_ZLIB_ERROR;\n \t\t\tgoto done;\n \t\t}\n \n-\t\tbr->zstream->next_in = br->block.data + block_header_skip;\n-\t\tbr->zstream->avail_in = src_len;\n-\t\tbr->zstream->next_out = br->uncompressed_data + block_header_skip;\n-\t\tbr->zstream->avail_out = dst_len;\n+\t\tblock->zstream->next_in = block->block.data + block_header_skip;\n+\t\tblock->zstream->avail_in = src_len;\n+\t\tblock->zstream->next_out = block->uncompressed_data + block_header_skip;\n+\t\tblock->zstream->avail_out = dst_len;\n \n \t\t/*\n \t\t * We know both input as well as output size, and we know that\n@@ -297,71 +297,71 @@ int block_reader_init(struct block_reader *br,\n \t\t * here to instruct zlib to inflate the data in one go, which\n \t\t * is more efficient than using `Z_NO_FLUSH`.\n \t\t */\n-\t\terr = inflate(br->zstream, Z_FINISH);\n+\t\terr = inflate(block->zstream, Z_FINISH);\n \t\tif (err != Z_STREAM_END) {\n \t\t\terr = REFTABLE_ZLIB_ERROR;\n \t\t\tgoto done;\n \t\t}\n \t\terr = 0;\n \n-\t\tif (br->zstream->total_out + block_header_skip != block_size) {\n+\t\tif (block->zstream->total_out + block_header_skip != block_size) {\n \t\t\terr = REFTABLE_FORMAT_ERROR;\n \t\t\tgoto done;\n \t\t}\n \n \t\t/* We're done with the input data. */\n-\t\tblock_source_release_data(&br->block);\n-\t\tbr->block.data = br->uncompressed_data;\n-\t\tbr->block.len = block_size;\n-\t\tfull_block_size = src_len + block_header_skip - br->zstream->avail_in;\n+\t\tblock_source_release_data(&block->block);\n+\t\tblock->block.data = block->uncompressed_data;\n+\t\tblock->block.len = block_size;\n+\t\tfull_block_size = src_len + block_header_skip - block->zstream->avail_in;\n \t} else if (full_block_size == 0) {\n \t\tfull_block_size = block_size;\n-\t} else if (block_size < full_block_size && block_size < br->block.len &&\n-\t\t   br->block.data[block_size] != 0) {\n+\t} else if (block_size < full_block_size && block_size < block->block.len &&\n+\t\t   block->block.data[block_size] != 0) {\n \t\t/* If the block is smaller than the full block size, it is\n \t\t   padded (data followed by '\\0') or the next block is\n \t\t   unaligned. */\n \t\tfull_block_size = block_size;\n \t}\n \n-\trestart_count = reftable_get_be16(br->block.data + block_size - 2);\n+\trestart_count = reftable_get_be16(block->block.data + block_size - 2);\n \trestart_off = block_size - 2 - 3 * restart_count;\n \n-\tbr->block_type = block_type;\n-\tbr->hash_size = hash_size;\n-\tbr->restart_off = restart_off;\n-\tbr->full_block_size = full_block_size;\n-\tbr->header_off = header_size;\n-\tbr->restart_count = restart_count;\n+\tblock->block_type = block_type;\n+\tblock->hash_size = hash_size;\n+\tblock->restart_off = restart_off;\n+\tblock->full_block_size = full_block_size;\n+\tblock->header_off = header_size;\n+\tblock->restart_count = restart_count;\n \n \terr = 0;\n \n done:\n \tif (err < 0)\n-\t\tblock_reader_release(br);\n+\t\treftable_block_release(block);\n \treturn err;\n }\n \n-void block_reader_release(struct block_reader *br)\n+void reftable_block_release(struct reftable_block *block)\n {\n-\tinflateEnd(br->zstream);\n-\treftable_free(br->zstream);\n-\treftable_free(br->uncompressed_data);\n-\tblock_source_release_data(&br->block);\n-\tmemset(br, 0, sizeof(*br));\n+\tinflateEnd(block->zstream);\n+\treftable_free(block->zstream);\n+\treftable_free(block->uncompressed_data);\n+\tblock_source_release_data(&block->block);\n+\tmemset(block, 0, sizeof(*block));\n }\n \n-uint8_t block_reader_type(const struct block_reader *r)\n+uint8_t reftable_block_type(const struct reftable_block *b)\n {\n-\treturn r->block.data[r->header_off];\n+\treturn b->block.data[b->header_off];\n }\n \n-int block_reader_first_key(const struct block_reader *br, struct reftable_buf *key)\n+int reftable_block_first_key(const struct reftable_block *block, struct reftable_buf *key)\n {\n-\tint off = br->header_off + 4, n;\n+\tint off = block->header_off + 4, n;\n \tstruct string_view in = {\n-\t\t.buf = br->block.data + off,\n-\t\t.len = br->restart_off - off,\n+\t\t.buf = block->block.data + off,\n+\t\t.len = block->restart_off - off,\n \t};\n \tuint8_t extra = 0;\n \n@@ -376,33 +376,33 @@ int block_reader_first_key(const struct block_reader *br, struct reftable_buf *k\n \treturn 0;\n }\n \n-static uint32_t block_reader_restart_offset(const struct block_reader *br, size_t idx)\n+static uint32_t block_restart_offset(const struct reftable_block *b, size_t idx)\n {\n-\treturn reftable_get_be24(br->block.data + br->restart_off + 3 * idx);\n+\treturn reftable_get_be24(b->block.data + b->restart_off + 3 * idx);\n }\n \n-void block_iter_seek_start(struct block_iter *it, const struct block_reader *br)\n+void block_iter_seek_start(struct block_iter *it, const struct reftable_block *b)\n {\n-\tit->block = br->block.data;\n-\tit->block_len = br->restart_off;\n-\tit->hash_size = br->hash_size;\n+\tit->block = b->block.data;\n+\tit->block_len = b->restart_off;\n+\tit->hash_size = b->hash_size;\n \treftable_buf_reset(&it->last_key);\n-\tit->next_off = br->header_off + 4;\n+\tit->next_off = b->header_off + 4;\n }\n \n struct restart_needle_less_args {\n \tint error;\n \tstruct reftable_buf needle;\n-\tconst struct block_reader *reader;\n+\tconst struct reftable_block *block;\n };\n \n static int restart_needle_less(size_t idx, void *_args)\n {\n \tstruct restart_needle_less_args *args = _args;\n-\tuint32_t off = block_reader_restart_offset(args->reader, idx);\n+\tuint32_t off = block_restart_offset(args->block, idx);\n \tstruct string_view in = {\n-\t\t.buf = args->reader->block.data + off,\n-\t\t.len = args->reader->restart_off - off,\n+\t\t.buf = args->block->block.data + off,\n+\t\t.len = args->block->restart_off - off,\n \t};\n \tuint64_t prefix_len, suffix_len;\n \tuint8_t extra;\n@@ -477,12 +477,12 @@ void block_iter_close(struct block_iter *it)\n \treftable_buf_release(&it->scratch);\n }\n \n-int block_iter_seek_key(struct block_iter *it, const struct block_reader *br,\n+int block_iter_seek_key(struct block_iter *it, const struct reftable_block *block,\n \t\t\tstruct reftable_buf *want)\n {\n \tstruct restart_needle_less_args args = {\n \t\t.needle = *want,\n-\t\t.reader = br,\n+\t\t.block = block,\n \t};\n \tstruct reftable_record rec;\n \tint err = 0;\n@@ -500,7 +500,7 @@ int block_iter_seek_key(struct block_iter *it, const struct block_reader *br,\n \t * restart point. While that works alright, we would end up scanning\n \t * too many record.\n \t */\n-\ti = binsearch(br->restart_count, &restart_needle_less, &args);\n+\ti = binsearch(block->restart_count, &restart_needle_less, &args);\n \tif (args.error) {\n \t\terr = REFTABLE_FORMAT_ERROR;\n \t\tgoto done;\n@@ -525,21 +525,21 @@ int block_iter_seek_key(struct block_iter *it, const struct block_reader *br,\n \t *     starting from the preceding restart point.\n \t */\n \tif (i > 0)\n-\t\tit->next_off = block_reader_restart_offset(br, i - 1);\n+\t\tit->next_off = block_restart_offset(block, i - 1);\n \telse\n-\t\tit->next_off = br->header_off + 4;\n-\tit->block = br->block.data;\n-\tit->block_len = br->restart_off;\n-\tit->hash_size = br->hash_size;\n+\t\tit->next_off = block->header_off + 4;\n+\tit->block = block->block.data;\n+\tit->block_len = block->restart_off;\n+\tit->hash_size = block->hash_size;\n \n-\terr = reftable_record_init(&rec, block_reader_type(br));\n+\terr = reftable_record_init(&rec, reftable_block_type(block));\n \tif (err < 0)\n \t\tgoto done;\n \n \t/*\n \t * We're looking for the last entry less than the wanted key so that\n \t * the next call to `block_reader_next()` would yield the wanted\n-\t * record. We thus don't want to position our reader at the sought\n+\t * record. We thus don't want to position our iterator at the sought\n \t * after record, but one before. To do so, we have to go one entry too\n \t * far and then back up.\n \t */\ndiff --git a/reftable/block.h b/reftable/block.h\nindex c55d47c3c52..880e2e61938 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -64,8 +64,11 @@ void block_writer_release(struct block_writer *bw);\n \n struct z_stream;\n \n-/* Read a block. */\n-struct block_reader {\n+/*\n+ * A block part of a reftable. Contains records as well as some metadata\n+ * describing them.\n+ */\n+struct reftable_block {\n \t/* offset of the block header; nonzero for the first block in a\n \t * reftable. */\n \tuint32_t header_off;\n@@ -92,19 +95,21 @@ struct block_reader {\n \tuint8_t block_type;\n };\n \n-/* initializes a block reader. */\n-int block_reader_init(struct block_reader *br,\n-\t\t      struct reftable_block_source *source,\n-\t\t      uint32_t offset, uint32_t header_size,\n-\t\t      uint32_t table_block_size, uint32_t hash_size);\n+/*\n+ * Initialize a reftable block from the given block source.\n+ */\n+int reftable_block_init(struct reftable_block *b,\n+\t\t\tstruct reftable_block_source *source,\n+\t\t\tuint32_t offset, uint32_t header_size,\n+\t\t\tuint32_t table_block_size, uint32_t hash_size);\n \n-void block_reader_release(struct block_reader *br);\n+void reftable_block_release(struct reftable_block *b);\n \n /* Returns the block type (eg. 'r' for refs) */\n-uint8_t block_reader_type(const struct block_reader *r);\n+uint8_t reftable_block_type(const struct reftable_block *b);\n \n /* Decodes the first key in the block */\n-int block_reader_first_key(const struct block_reader *br, struct reftable_buf *key);\n+int reftable_block_first_key(const struct reftable_block *b, struct reftable_buf *key);\n \n /* Iterate over entries in a block */\n struct block_iter {\n@@ -125,10 +130,10 @@ struct block_iter {\n }\n \n /* Position `it` at start of the block */\n-void block_iter_seek_start(struct block_iter *it, const struct block_reader *br);\n+void block_iter_seek_start(struct block_iter *it, const struct reftable_block *block);\n \n /* Position `it` to the `want` key in the block */\n-int block_iter_seek_key(struct block_iter *it, const struct block_reader *br,\n+int block_iter_seek_key(struct block_iter *it, const struct reftable_block *block,\n \t\t\tstruct reftable_buf *want);\n \n /* return < 0 for error, 0 for OK, > 0 for EOF. */\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex 37b354a6fb9..74684de8f6a 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -114,7 +114,7 @@ static void indexed_table_ref_iter_close(void *p)\n {\n \tstruct indexed_table_ref_iter *it = p;\n \tblock_iter_close(&it->cur);\n-\tblock_source_release_data(&it->block_reader.block);\n+\tblock_source_release_data(&it->block.block);\n \treftable_free(it->offsets);\n \treftable_buf_release(&it->oid);\n }\n@@ -128,11 +128,10 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n \t\treturn 1;\n \t}\n \n-\tblock_source_release_data(&it->block_reader.block);\n+\tblock_source_release_data(&it->block.block);\n \n \toff = it->offsets[it->offset_idx++];\n-\terr = table_init_block_reader(it->table, &it->block_reader, off,\n-\t\t\t\t      BLOCK_TYPE_REF);\n+\terr = table_init_block(it->table, &it->block, off, BLOCK_TYPE_REF);\n \tif (err < 0) {\n \t\treturn err;\n \t}\n@@ -140,7 +139,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n \t\t/* indexed block does not exist. */\n \t\treturn REFTABLE_FORMAT_ERROR;\n \t}\n-\tblock_iter_seek_start(&it->cur, &it->block_reader);\n+\tblock_iter_seek_start(&it->cur, &it->block);\n \treturn 0;\n }\n \ndiff --git a/reftable/iter.h b/reftable/iter.h\nindex add5c9472fc..cc920970a57 100644\n--- a/reftable/iter.h\n+++ b/reftable/iter.h\n@@ -68,7 +68,7 @@ struct indexed_table_ref_iter {\n \t/* Points to the next offset to read. */\n \tint offset_idx;\n \tint offset_len;\n-\tstruct block_reader block_reader;\n+\tstruct reftable_block block;\n \tstruct block_iter cur;\n \tint is_finished;\n };\ndiff --git a/reftable/table.c b/reftable/table.c\nindex a5ce96fa51c..3f2e70caac4 100644\n--- a/reftable/table.c\n+++ b/reftable/table.c\n@@ -131,7 +131,7 @@ struct table_iter {\n \tstruct reftable_table *table;\n \tuint8_t typ;\n \tuint64_t block_off;\n-\tstruct block_reader br;\n+\tstruct reftable_block block;\n \tstruct block_iter bi;\n \tint is_finished;\n };\n@@ -159,12 +159,12 @@ static int table_iter_next_in_block(struct table_iter *ti,\n \n static void table_iter_block_done(struct table_iter *ti)\n {\n-\tblock_reader_release(&ti->br);\n+\treftable_block_release(&ti->block);\n \tblock_iter_reset(&ti->bi);\n }\n \n-int table_init_block_reader(struct reftable_table *t, struct block_reader *br,\n-\t\t\t    uint64_t next_off, uint8_t want_typ)\n+int table_init_block(struct reftable_table *t, struct reftable_block *block,\n+\t\t     uint64_t next_off, uint8_t want_typ)\n {\n \tuint32_t header_off = next_off ? 0 : header_size(t->version);\n \tint err;\n@@ -172,19 +172,19 @@ int table_init_block_reader(struct reftable_table *t, struct block_reader *br,\n \tif (next_off >= t->size)\n \t\treturn 1;\n \n-\terr = block_reader_init(br, &t->source, next_off, header_off,\n-\t\t\t\tt->block_size, hash_size(t->hash_id));\n+\terr = reftable_block_init(block, &t->source, next_off, header_off,\n+\t\t\t\t  t->block_size, hash_size(t->hash_id));\n \tif (err < 0)\n \t\tgoto done;\n \n-\tif (want_typ != BLOCK_TYPE_ANY && br->block_type != want_typ) {\n+\tif (want_typ != BLOCK_TYPE_ANY && block->block_type != want_typ) {\n \t\terr = 1;\n \t\tgoto done;\n \t}\n \n done:\n \tif (err)\n-\t\tblock_reader_release(br);\n+\t\treftable_block_release(block);\n \treturn err;\n }\n \n@@ -197,10 +197,10 @@ static void table_iter_close(struct table_iter *ti)\n \n static int table_iter_next_block(struct table_iter *ti)\n {\n-\tuint64_t next_block_off = ti->block_off + ti->br.full_block_size;\n+\tuint64_t next_block_off = ti->block_off + ti->block.full_block_size;\n \tint err;\n \n-\terr = table_init_block_reader(ti->table, &ti->br, next_block_off, ti->typ);\n+\terr = table_init_block(ti->table, &ti->block, next_block_off, ti->typ);\n \tif (err > 0)\n \t\tti->is_finished = 1;\n \tif (err)\n@@ -208,7 +208,7 @@ static int table_iter_next_block(struct table_iter *ti)\n \n \tti->block_off = next_block_off;\n \tti->is_finished = 0;\n-\tblock_iter_seek_start(&ti->bi, &ti->br);\n+\tblock_iter_seek_start(&ti->bi, &ti->block);\n \n \treturn 0;\n }\n@@ -250,13 +250,13 @@ static int table_iter_seek_to(struct table_iter *ti, uint64_t off, uint8_t typ)\n {\n \tint err;\n \n-\terr = table_init_block_reader(ti->table, &ti->br, off, typ);\n+\terr = table_init_block(ti->table, &ti->block, off, typ);\n \tif (err != 0)\n \t\treturn err;\n \n-\tti->typ = block_reader_type(&ti->br);\n+\tti->typ = reftable_block_type(&ti->block);\n \tti->block_off = off;\n-\tblock_iter_seek_start(&ti->bi, &ti->br);\n+\tblock_iter_seek_start(&ti->bi, &ti->block);\n \tti->is_finished = 0;\n \treturn 0;\n }\n@@ -320,10 +320,10 @@ static int table_iter_seek_linear(struct table_iter *ti,\n \t\t * as we have more than three blocks we would have an index, so\n \t\t * we would not do a linear search there anymore.\n \t\t */\n-\t\tmemset(&next.br.block, 0, sizeof(next.br.block));\n-\t\tnext.br.zstream = NULL;\n-\t\tnext.br.uncompressed_data = NULL;\n-\t\tnext.br.uncompressed_cap = 0;\n+\t\tmemset(&next.block.block, 0, sizeof(next.block.block));\n+\t\tnext.block.zstream = NULL;\n+\t\tnext.block.uncompressed_data = NULL;\n+\t\tnext.block.uncompressed_cap = 0;\n \n \t\terr = table_iter_next_block(&next);\n \t\tif (err < 0)\n@@ -331,7 +331,7 @@ static int table_iter_seek_linear(struct table_iter *ti,\n \t\tif (err > 0)\n \t\t\tbreak;\n \n-\t\terr = block_reader_first_key(&next.br, &got_key);\n+\t\terr = reftable_block_first_key(&next.block, &got_key);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n@@ -349,7 +349,7 @@ static int table_iter_seek_linear(struct table_iter *ti,\n \t * the wanted key inside of it. If the block does not contain our key\n \t * we know that the corresponding record does not exist.\n \t */\n-\terr = block_iter_seek_key(&ti->bi, &ti->br, &want_key);\n+\terr = block_iter_seek_key(&ti->bi, &ti->block, &want_key);\n \tif (err < 0)\n \t\tgoto done;\n \terr = 0;\n@@ -417,7 +417,7 @@ static int table_iter_seek_indexed(struct table_iter *ti,\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = block_iter_seek_key(&ti->bi, &ti->br, &want_index.u.idx.last_key);\n+\t\terr = block_iter_seek_key(&ti->bi, &ti->block, &want_index.u.idx.last_key);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n@@ -785,8 +785,8 @@ int reftable_table_print_blocks(const char *tablename)\n \t\tprintf(\"%s:\\n\", sections[i].name);\n \n \t\twhile (1) {\n-\t\t\tprintf(\"  - length: %u\\n\", ti.br.restart_off);\n-\t\t\tprintf(\"    restarts: %u\\n\", ti.br.restart_count);\n+\t\t\tprintf(\"  - length: %u\\n\", ti.block.restart_off);\n+\t\t\tprintf(\"    restarts: %u\\n\", ti.block.restart_count);\n \n \t\t\terr = table_iter_next_block(&ti);\n \t\t\tif (err < 0)\ndiff --git a/reftable/table.h b/reftable/table.h\nindex 8d8dd2b413d..e15d58d8e9e 100644\n--- a/reftable/table.h\n+++ b/reftable/table.h\n@@ -53,8 +53,10 @@ int table_init_iter(struct reftable_table *t,\n \t\t    struct reftable_iterator *it,\n \t\t    uint8_t typ);\n \n-/* initialize a block reader to read from `t` */\n-int table_init_block_reader(struct reftable_table *t, struct block_reader *br,\n-\t\t\t    uint64_t next_off, uint8_t want_typ);\n+/*\n+ * Initialize a block by reading from the given table and offset.\n+ */\n+int table_init_block(struct reftable_table *t, struct reftable_block *block,\n+\t\t     uint64_t next_off, uint8_t want_typ);\n \n #endif\ndiff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c\nindex dbec6a5d0d0..e36ed7ac576 100644\n--- a/t/unit-tests/t-reftable-block.c\n+++ b/t/unit-tests/t-reftable-block.c\n@@ -28,16 +28,16 @@ static void t_ref_block_read_write(void)\n \t};\n \tsize_t i = 0;\n \tint ret;\n-\tstruct block_reader br = { 0 };\n+\tstruct reftable_block block = { 0 };\n \tstruct block_iter it = BLOCK_ITER_INIT;\n \tstruct reftable_buf want = REFTABLE_BUF_INIT;\n-\tstruct reftable_buf block = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf block_data = REFTABLE_BUF_INIT;\n \n-\tREFTABLE_CALLOC_ARRAY(block.buf, block_size);\n-\tcheck(block.buf != NULL);\n-\tblock.len = block_size;\n+\tREFTABLE_CALLOC_ARRAY(block_data.buf, block_size);\n+\tcheck(block_data.buf != NULL);\n+\tblock_data.len = block_size;\n \n-\tret = block_writer_init(&bw, BLOCK_TYPE_REF, (uint8_t *) block.buf, block_size,\n+\tret = block_writer_init(&bw, BLOCK_TYPE_REF, (uint8_t *) block_data.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -63,10 +63,10 @@ static void t_ref_block_read_write(void)\n \n \tblock_writer_release(&bw);\n \n-\tblock_source_from_buf(&source ,&block);\n-\tblock_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n+\tblock_source_from_buf(&source ,&block_data);\n+\treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n-\tblock_iter_seek_start(&it, &br);\n+\tblock_iter_seek_start(&it, &block);\n \n \tfor (i = 0; ; i++) {\n \t\tret = block_iter_next(&it, &rec);\n@@ -82,7 +82,7 @@ static void t_ref_block_read_write(void)\n \t\tblock_iter_reset(&it);\n \t\treftable_record_key(&recs[i], &want);\n \n-\t\tret = block_iter_seek_key(&it, &br, &want);\n+\t\tret = block_iter_seek_key(&it, &block, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -91,7 +91,7 @@ static void t_ref_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1));\n \n \t\twant.len--;\n-\t\tret = block_iter_seek_key(&it, &br, &want);\n+\t\tret = block_iter_seek_key(&it, &block, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -99,11 +99,11 @@ static void t_ref_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[10 * (i / 10)], &rec, REFTABLE_HASH_SIZE_SHA1));\n \t}\n \n-\tblock_reader_release(&br);\n+\treftable_block_release(&block);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n \treftable_buf_release(&want);\n-\treftable_buf_release(&block);\n+\treftable_buf_release(&block_data);\n \tfor (i = 0; i < N; i++)\n \t\treftable_record_release(&recs[i]);\n }\n@@ -123,16 +123,16 @@ static void t_log_block_read_write(void)\n \t};\n \tsize_t i = 0;\n \tint ret;\n-\tstruct block_reader br = { 0 };\n+\tstruct reftable_block block = { 0 };\n \tstruct block_iter it = BLOCK_ITER_INIT;\n \tstruct reftable_buf want = REFTABLE_BUF_INIT;\n-\tstruct reftable_buf block = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf block_data = REFTABLE_BUF_INIT;\n \n-\tREFTABLE_CALLOC_ARRAY(block.buf, block_size);\n-\tcheck(block.buf != NULL);\n-\tblock.len = block_size;\n+\tREFTABLE_CALLOC_ARRAY(block_data.buf, block_size);\n+\tcheck(block_data.buf != NULL);\n+\tblock_data.len = block_size;\n \n-\tret = block_writer_init(&bw, BLOCK_TYPE_LOG, (uint8_t *) block.buf, block_size,\n+\tret = block_writer_init(&bw, BLOCK_TYPE_LOG, (uint8_t *) block_data.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -153,10 +153,10 @@ static void t_log_block_read_write(void)\n \n \tblock_writer_release(&bw);\n \n-\tblock_source_from_buf(&source, &block);\n-\tblock_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n+\tblock_source_from_buf(&source, &block_data);\n+\treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n-\tblock_iter_seek_start(&it, &br);\n+\tblock_iter_seek_start(&it, &block);\n \n \tfor (i = 0; ; i++) {\n \t\tret = block_iter_next(&it, &rec);\n@@ -173,7 +173,7 @@ static void t_log_block_read_write(void)\n \t\treftable_buf_reset(&want);\n \t\tcheck(!reftable_buf_addstr(&want, recs[i].u.log.refname));\n \n-\t\tret = block_iter_seek_key(&it, &br, &want);\n+\t\tret = block_iter_seek_key(&it, &block, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -182,7 +182,7 @@ static void t_log_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1));\n \n \t\twant.len--;\n-\t\tret = block_iter_seek_key(&it, &br, &want);\n+\t\tret = block_iter_seek_key(&it, &block, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -190,11 +190,11 @@ static void t_log_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[10 * (i / 10)], &rec, REFTABLE_HASH_SIZE_SHA1));\n \t}\n \n-\tblock_reader_release(&br);\n+\treftable_block_release(&block);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n \treftable_buf_release(&want);\n-\treftable_buf_release(&block);\n+\treftable_buf_release(&block_data);\n \tfor (i = 0; i < N; i++)\n \t\treftable_record_release(&recs[i]);\n }\n@@ -214,16 +214,16 @@ static void t_obj_block_read_write(void)\n \t};\n \tsize_t i = 0;\n \tint ret;\n-\tstruct block_reader br = { 0 };\n+\tstruct reftable_block block = { 0 };\n \tstruct block_iter it = BLOCK_ITER_INIT;\n \tstruct reftable_buf want = REFTABLE_BUF_INIT;\n-\tstruct reftable_buf block = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf block_data = REFTABLE_BUF_INIT;\n \n-\tREFTABLE_CALLOC_ARRAY(block.buf, block_size);\n-\tcheck(block.buf != NULL);\n-\tblock.len = block_size;\n+\tREFTABLE_CALLOC_ARRAY(block_data.buf, block_size);\n+\tcheck(block_data.buf != NULL);\n+\tblock_data.len = block_size;\n \n-\tret = block_writer_init(&bw, BLOCK_TYPE_OBJ, (uint8_t *) block.buf, block_size,\n+\tret = block_writer_init(&bw, BLOCK_TYPE_OBJ, (uint8_t *) block_data.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -246,10 +246,10 @@ static void t_obj_block_read_write(void)\n \n \tblock_writer_release(&bw);\n \n-\tblock_source_from_buf(&source, &block);\n-\tblock_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n+\tblock_source_from_buf(&source, &block_data);\n+\treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n-\tblock_iter_seek_start(&it, &br);\n+\tblock_iter_seek_start(&it, &block);\n \n \tfor (i = 0; ; i++) {\n \t\tret = block_iter_next(&it, &rec);\n@@ -265,7 +265,7 @@ static void t_obj_block_read_write(void)\n \t\tblock_iter_reset(&it);\n \t\treftable_record_key(&recs[i], &want);\n \n-\t\tret = block_iter_seek_key(&it, &br, &want);\n+\t\tret = block_iter_seek_key(&it, &block, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -274,11 +274,11 @@ static void t_obj_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1));\n \t}\n \n-\tblock_reader_release(&br);\n+\treftable_block_release(&block);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n \treftable_buf_release(&want);\n-\treftable_buf_release(&block);\n+\treftable_buf_release(&block_data);\n \tfor (i = 0; i < N; i++)\n \t\treftable_record_release(&recs[i]);\n }\n@@ -299,16 +299,16 @@ static void t_index_block_read_write(void)\n \t};\n \tsize_t i = 0;\n \tint ret;\n-\tstruct block_reader br = { 0 };\n+\tstruct reftable_block block = { 0 };\n \tstruct block_iter it = BLOCK_ITER_INIT;\n \tstruct reftable_buf want = REFTABLE_BUF_INIT;\n-\tstruct reftable_buf block = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf block_data = REFTABLE_BUF_INIT;\n \n-\tREFTABLE_CALLOC_ARRAY(block.buf, block_size);\n-\tcheck(block.buf != NULL);\n-\tblock.len = block_size;\n+\tREFTABLE_CALLOC_ARRAY(block_data.buf, block_size);\n+\tcheck(block_data.buf != NULL);\n+\tblock_data.len = block_size;\n \n-\tret = block_writer_init(&bw, BLOCK_TYPE_INDEX, (uint8_t *) block.buf, block_size,\n+\tret = block_writer_init(&bw, BLOCK_TYPE_INDEX, (uint8_t *) block_data.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -331,10 +331,10 @@ static void t_index_block_read_write(void)\n \n \tblock_writer_release(&bw);\n \n-\tblock_source_from_buf(&source, &block);\n-\tblock_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n+\tblock_source_from_buf(&source, &block_data);\n+\treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n-\tblock_iter_seek_start(&it, &br);\n+\tblock_iter_seek_start(&it, &block);\n \n \tfor (i = 0; ; i++) {\n \t\tret = block_iter_next(&it, &rec);\n@@ -350,7 +350,7 @@ static void t_index_block_read_write(void)\n \t\tblock_iter_reset(&it);\n \t\treftable_record_key(&recs[i], &want);\n \n-\t\tret = block_iter_seek_key(&it, &br, &want);\n+\t\tret = block_iter_seek_key(&it, &block, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -359,7 +359,7 @@ static void t_index_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1));\n \n \t\twant.len--;\n-\t\tret = block_iter_seek_key(&it, &br, &want);\n+\t\tret = block_iter_seek_key(&it, &block, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -367,11 +367,11 @@ static void t_index_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[10 * (i / 10)], &rec, REFTABLE_HASH_SIZE_SHA1));\n \t}\n \n-\tblock_reader_release(&br);\n+\treftable_block_release(&block);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n \treftable_buf_release(&want);\n-\treftable_buf_release(&block);\n+\treftable_buf_release(&block_data);\n \tfor (i = 0; i < N; i++)\n \t\treftable_record_release(&recs[i]);\n }\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515345","messageId":"20250331-pks-reftable-polishing-v1-8-ebed5247434c@pks.im","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im","subject":"[PATCH 08/16] git-zlib: use `struct z_stream_s` instead of typedef","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-31T08:41:26Z","receivedAt":"2025-03-31T08:41:30Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Throughout the Git codebase we're using the typedeffed version of\n`z_stream`, which maps to `struct z_stream_s`. By using a typedef\ninstead of the struct it becomes somewhat harder to predeclare the\nsymbol so that headers depending on the struct can do so without having\nto pull in \"zlib-compat.h\".\n\nWe don't yet have users that would really care about this: the only\nusers that declare `z_stream` as a pointer are in \"reftable/block.h\",\nwhich is a header that is internal to the reftable library. But in the\nnext step we're going to expose the `struct reftable_block` publicly,\nand that struct does contain a pointer to `z_stream`. And as the public\nheader shouldn't depend on \"reftable/system.h\", which is an internal\nimplementation detail, we won't have the typedef for `z_stream` readily\navailable.\n\nPrepare for this change by using `struct z_stream_s` throughout our code\nbase. In case zlib-ng is used we use a define to map from `z_stream_s`\nto `zng_stream_s`.\n\nDrop the pre-declaration of `struct z_stream` while at it. This struct\ndoes not exist in the first place, and the declaration wasn't needed\nbecause \"reftable/block.h\" already includes \"reftable/basics.h\" which\ntransitively includes \"reftable/system.h\" and thus \"git-zlib.h\".\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n Documentation/howto/recover-corrupted-object-harder.adoc | 4 ++--\n compat/zlib-compat.h                                     | 4 ++--\n git-zlib.h                                               | 2 +-\n reftable/block.h                                         | 6 ++----\n 4 files changed, 7 insertions(+), 9 deletions(-)\n\ndiff --git a/Documentation/howto/recover-corrupted-object-harder.adoc b/Documentation/howto/recover-corrupted-object-harder.adoc\nindex 5efb4fe81ff..86a1ba75cf9 100644\n--- a/Documentation/howto/recover-corrupted-object-harder.adoc\n+++ b/Documentation/howto/recover-corrupted-object-harder.adoc\n@@ -125,7 +125,7 @@ static int try_zlib(unsigned char *buf, int len)\n {\n \t/* make this absurdly large so we don't have to loop */\n \tstatic unsigned char out[1024*1024];\n-\tz_stream z;\n+\tstruct z_stream_s z;\n \tint ret;\n \n \tmemset(&z, 0, sizeof(z));\n@@ -278,7 +278,7 @@ int main(int argc, char **argv)\n \tstatic unsigned char buf[25 * 1024 * 1024];\n \tstatic unsigned char out[25 * 1024 * 1024];\n \tint len;\n-\tz_stream z;\n+\tstruct z_stream_s z;\n \tint ret;\n \n \tlen = read(0, buf, sizeof(buf));\ndiff --git a/compat/zlib-compat.h b/compat/zlib-compat.h\nindex 0c60e3af331..ac082766229 100644\n--- a/compat/zlib-compat.h\n+++ b/compat/zlib-compat.h\n@@ -4,8 +4,8 @@\n #ifdef HAVE_ZLIB_NG\n # include <zlib-ng.h>\n \n-# define z_stream zng_stream\n-#define gz_header_s zng_gz_header_s\n+# define z_stream_s zng_stream_s\n+# define gz_header_s zng_gz_header_s\n \n # define crc32(crc, buf, len) zng_crc32(crc, buf, len)\n \ndiff --git a/git-zlib.h b/git-zlib.h\nindex 1e8d9aabcb4..0e66fefa8c9 100644\n--- a/git-zlib.h\n+++ b/git-zlib.h\n@@ -4,7 +4,7 @@\n #include \"compat/zlib-compat.h\"\n \n typedef struct git_zstream {\n-\tz_stream z;\n+\tstruct z_stream_s z;\n \tunsigned long avail_in;\n \tunsigned long avail_out;\n \tunsigned long total_in;\ndiff --git a/reftable/block.h b/reftable/block.h\nindex 880e2e61938..75bc10f63a3 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -18,7 +18,7 @@\n  * allocation overhead.\n  */\n struct block_writer {\n-\tz_stream *zstream;\n+\tstruct z_stream_s *zstream;\n \tunsigned char *compressed;\n \tsize_t compressed_cap;\n \n@@ -62,8 +62,6 @@ int block_writer_finish(struct block_writer *w);\n /* clears out internally allocated block_writer members. */\n void block_writer_release(struct block_writer *bw);\n \n-struct z_stream;\n-\n /*\n  * A block part of a reftable. Contains records as well as some metadata\n  * describing them.\n@@ -78,7 +76,7 @@ struct reftable_block {\n \tuint32_t hash_size;\n \n \t/* Uncompressed data for log entries. */\n-\tz_stream *zstream;\n+\tstruct z_stream_s *zstream;\n \tunsigned char *uncompressed_data;\n \tsize_t uncompressed_cap;\n \n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515347","messageId":"20250331-pks-reftable-polishing-v1-10-ebed5247434c@pks.im","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im","subject":"[PATCH 10/16] reftable/block: store block pointer in the block iterator","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-31T08:41:28Z","receivedAt":"2025-03-31T08:41:31Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The block iterator requires access to a bunch of data from the\nunderlying `reftable_block` that it is iterating over. This data is\nstored by copying over relevant data into a separate set of variables.\nThis has multiple downsides:\n\n  - We require more storage space than necessary. This is more of a\n    theoretical issue as we shouldn't ever have many blocks.\n\n  - We have to perform more bookkeeping, and the variable names are\n    inconsistent across the two data structures. This can lead to some\n    confusion.\n\n  - The lifetime of the block iterator is tied to the block anyway, but\n    we hide that a bit by only storing pointers into the block.\n\nThere isn't really any good reason why we rip out parts of the block\ninstead of storing a pointer to the block itself.\n\nRefactor the code to do so. Despite being simpler, it also allows us to\ndecouple the lifetime of the block iterator from seeking in a subsequent\ncommit.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/block.c | 22 ++++++++--------------\n reftable/block.h |  4 +---\n 2 files changed, 9 insertions(+), 17 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex d188665388d..576c6caf59b 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -381,13 +381,11 @@ static uint32_t block_restart_offset(const struct reftable_block *b, size_t idx)\n \treturn reftable_get_be24(b->block.data + b->restart_off + 3 * idx);\n }\n \n-void block_iter_seek_start(struct block_iter *it, const struct reftable_block *b)\n+void block_iter_seek_start(struct block_iter *it, const struct reftable_block *block)\n {\n-\tit->block = b->block.data;\n-\tit->block_len = b->restart_off;\n-\tit->hash_size = b->hash_size;\n+\tit->block = block;\n \treftable_buf_reset(&it->last_key);\n-\tit->next_off = b->header_off + 4;\n+\tit->next_off = block->header_off + 4;\n }\n \n struct restart_needle_less_args {\n@@ -435,14 +433,14 @@ static int restart_needle_less(size_t idx, void *_args)\n int block_iter_next(struct block_iter *it, struct reftable_record *rec)\n {\n \tstruct string_view in = {\n-\t\t.buf = (unsigned char *) it->block + it->next_off,\n-\t\t.len = it->block_len - it->next_off,\n+\t\t.buf = (unsigned char *) it->block->block.data + it->next_off,\n+\t\t.len = it->block->restart_off - it->next_off,\n \t};\n \tstruct string_view start = in;\n \tuint8_t extra = 0;\n \tint n = 0;\n \n-\tif (it->next_off >= it->block_len)\n+\tif (it->next_off >= it->block->restart_off)\n \t\treturn 1;\n \n \tn = reftable_decode_key(&it->last_key, &extra, in);\n@@ -452,7 +450,7 @@ int block_iter_next(struct block_iter *it, struct reftable_record *rec)\n \t\treturn REFTABLE_FORMAT_ERROR;\n \n \tstring_view_consume(&in, n);\n-\tn = reftable_record_decode(rec, it->last_key, extra, in, it->hash_size,\n+\tn = reftable_record_decode(rec, it->last_key, extra, in, it->block->hash_size,\n \t\t\t\t   &it->scratch);\n \tif (n < 0)\n \t\treturn -1;\n@@ -467,8 +465,6 @@ void block_iter_reset(struct block_iter *it)\n \treftable_buf_reset(&it->last_key);\n \tit->next_off = 0;\n \tit->block = NULL;\n-\tit->block_len = 0;\n-\tit->hash_size = 0;\n }\n \n void block_iter_close(struct block_iter *it)\n@@ -528,9 +524,7 @@ int block_iter_seek_key(struct block_iter *it, const struct reftable_block *bloc\n \t\tit->next_off = block_restart_offset(block, i - 1);\n \telse\n \t\tit->next_off = block->header_off + 4;\n-\tit->block = block->block.data;\n-\tit->block_len = block->restart_off;\n-\tit->hash_size = block->hash_size;\n+\tit->block = block;\n \n \terr = reftable_record_init(&rec, reftable_block_type(block));\n \tif (err < 0)\ndiff --git a/reftable/block.h b/reftable/block.h\nindex 4f7f29028c4..268d5a1e005 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -67,9 +67,7 @@ void block_writer_release(struct block_writer *bw);\n struct block_iter {\n \t/* offset within the block of the next entry to read. */\n \tuint32_t next_off;\n-\tconst unsigned char *block;\n-\tsize_t block_len;\n-\tuint32_t hash_size;\n+\tconst struct reftable_block *block;\n \n \t/* key for last entry we read. */\n \tstruct reftable_buf last_key;\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515348","messageId":"20250331-pks-reftable-polishing-v1-9-ebed5247434c@pks.im","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im","subject":"[PATCH 09/16] reftable/block: create public interface for reading blocks","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-31T08:41:27Z","receivedAt":"2025-03-31T08:41:31Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"While users of the reftable library wouldn't generally require access to\nindividual blocks in a reftable table, there are valid usecases where\none may require low-level access to them. One such upcoming usecase in\nthe Git codebase is to implement consistency checks for the reftable\nlibrary where we want to verify each block individually.\n\nCreate a public interface for reading blocks. The interface isn't yet\ncomplete and lacks e.g. a way to read individual records from a block.\nSuch missing functionality will be backfilled in subsequent commits.\n\nNote that this change also requires us to expose `reftable_buf`, which\nis used by the `reftable_block_first_key()` function.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/basics.h          |  7 -----\n reftable/block.h           | 48 +-------------------------------\n reftable/reftable-basics.h |  8 ++++++\n reftable/reftable-block.h  | 69 ++++++++++++++++++++++++++++++++++++++++++++++\n 4 files changed, 78 insertions(+), 54 deletions(-)\n\ndiff --git a/reftable/basics.h b/reftable/basics.h\nindex 96a2f0d3829..d8888c12629 100644\n--- a/reftable/basics.h\n+++ b/reftable/basics.h\n@@ -18,13 +18,6 @@\n \n #define REFTABLE_UNUSED __attribute__((__unused__))\n \n-struct reftable_buf {\n-\tsize_t alloc;\n-\tsize_t len;\n-\tchar *buf;\n-};\n-#define REFTABLE_BUF_INIT { 0 }\n-\n /*\n  * Initialize the buffer such that it is ready for use. This is equivalent to\n  * using REFTABLE_BUF_INIT for stack-allocated variables.\ndiff --git a/reftable/block.h b/reftable/block.h\nindex 75bc10f63a3..4f7f29028c4 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -11,6 +11,7 @@\n \n #include \"basics.h\"\n #include \"record.h\"\n+#include \"reftable-block.h\"\n #include \"reftable-blocksource.h\"\n \n /*\n@@ -62,53 +63,6 @@ int block_writer_finish(struct block_writer *w);\n /* clears out internally allocated block_writer members. */\n void block_writer_release(struct block_writer *bw);\n \n-/*\n- * A block part of a reftable. Contains records as well as some metadata\n- * describing them.\n- */\n-struct reftable_block {\n-\t/* offset of the block header; nonzero for the first block in a\n-\t * reftable. */\n-\tuint32_t header_off;\n-\n-\t/* the memory block */\n-\tstruct reftable_block_data block;\n-\tuint32_t hash_size;\n-\n-\t/* Uncompressed data for log entries. */\n-\tstruct z_stream_s *zstream;\n-\tunsigned char *uncompressed_data;\n-\tsize_t uncompressed_cap;\n-\n-\t/*\n-\t * Restart point data. Restart points are located after the block's\n-\t * record data.\n-\t */\n-\tuint16_t restart_count;\n-\tuint32_t restart_off;\n-\n-\t/* size of the data in the file. For log blocks, this is the compressed\n-\t * size. */\n-\tuint32_t full_block_size;\n-\tuint8_t block_type;\n-};\n-\n-/*\n- * Initialize a reftable block from the given block source.\n- */\n-int reftable_block_init(struct reftable_block *b,\n-\t\t\tstruct reftable_block_source *source,\n-\t\t\tuint32_t offset, uint32_t header_size,\n-\t\t\tuint32_t table_block_size, uint32_t hash_size);\n-\n-void reftable_block_release(struct reftable_block *b);\n-\n-/* Returns the block type (eg. 'r' for refs) */\n-uint8_t reftable_block_type(const struct reftable_block *b);\n-\n-/* Decodes the first key in the block */\n-int reftable_block_first_key(const struct reftable_block *b, struct reftable_buf *key);\n-\n /* Iterate over entries in a block */\n struct block_iter {\n \t/* offset within the block of the next entry to read. */\ndiff --git a/reftable/reftable-basics.h b/reftable/reftable-basics.h\nindex ed7c7c9ac2c..6d73f19c85b 100644\n--- a/reftable/reftable-basics.h\n+++ b/reftable/reftable-basics.h\n@@ -11,6 +11,14 @@\n \n #include <stddef.h>\n \n+/* A buffer that contains arbitrary byte slices. */\n+struct reftable_buf {\n+\tsize_t alloc;\n+\tsize_t len;\n+\tchar *buf;\n+};\n+#define REFTABLE_BUF_INIT { 0 }\n+\n /*\n  * Hash functions understood by the reftable library. Note that the values are\n  * arbitrary and somewhat random such that we can easily detect cases where the\ndiff --git a/reftable/reftable-block.h b/reftable/reftable-block.h\nnew file mode 100644\nindex 00000000000..ce1d9d3ce79\n--- /dev/null\n+++ b/reftable/reftable-block.h\n@@ -0,0 +1,69 @@\n+/*\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n+\n+#ifndef REFTABLE_BLOCK_H\n+#define REFTABLE_BLOCK_H\n+\n+#include <stdint.h>\n+\n+#include \"reftable-basics.h\"\n+#include \"reftable-blocksource.h\"\n+\n+struct z_stream_s;\n+\n+/*\n+ * A block part of a reftable. Contains records as well as some metadata\n+ * describing them.\n+ */\n+struct reftable_block {\n+\t/*\n+\t * Offset of the block header; nonzero for the first block in a\n+\t * reftable.\n+\t */\n+\tuint32_t header_off;\n+\n+\t/* The memory block. */\n+\tstruct reftable_block_data block;\n+\tuint32_t hash_size;\n+\n+\t/* Uncompressed data for log entries. */\n+\tstruct z_stream_s *zstream;\n+\tunsigned char *uncompressed_data;\n+\tsize_t uncompressed_cap;\n+\n+\t/*\n+\t * Restart point data. Restart points are located after the block's\n+\t * record data.\n+\t */\n+\tuint16_t restart_count;\n+\tuint32_t restart_off;\n+\n+\t/*\n+\t * Size of the data in the file. For log blocks, this is the compressed\n+\t * size.\n+\t */\n+\tuint32_t full_block_size;\n+\tuint8_t block_type;\n+};\n+\n+/* Initialize a reftable block from the given block source. */\n+int reftable_block_init(struct reftable_block *b,\n+\t\t\tstruct reftable_block_source *source,\n+\t\t\tuint32_t offset, uint32_t header_size,\n+\t\t\tuint32_t table_block_size, uint32_t hash_size);\n+\n+/* Release resources allocated by the block. */\n+void reftable_block_release(struct reftable_block *b);\n+\n+/* Returns the block type (eg. 'r' for refs). */\n+uint8_t reftable_block_type(const struct reftable_block *b);\n+\n+/* Decodes the first key in the block. */\n+int reftable_block_first_key(const struct reftable_block *b, struct reftable_buf *key);\n+\n+#endif /* REFTABLE_BLOCK_H */\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515349","messageId":"20250331-pks-reftable-polishing-v1-11-ebed5247434c@pks.im","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im","subject":"[PATCH 11/16] reftable/block: make block iterators reseekable","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-31T08:41:29Z","receivedAt":"2025-03-31T08:41:32Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Refactor the block iterators so that initialization and seeking are\ndifferent from one another. This makes the iterator trivially reseekable\nby storing the pointer to the block at initialization time, which we can\nthen reuse on ever seek.\n\nThis refactoring prepares the code for exposing a `reftable_iterator`\ninterface for blocks in a subsequent commit. Callsites are adjusted\naccordingly.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/block.c                | 23 +++++++++++++----------\n reftable/block.h                | 21 ++++++++++++++++-----\n reftable/iter.c                 |  2 +-\n reftable/table.c                | 11 +++++++----\n t/unit-tests/t-reftable-block.c | 30 +++++++++++++++---------------\n 5 files changed, 52 insertions(+), 35 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 576c6caf59b..6e5c1191f5f 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -381,11 +381,16 @@ static uint32_t block_restart_offset(const struct reftable_block *b, size_t idx)\n \treturn reftable_get_be24(b->block.data + b->restart_off + 3 * idx);\n }\n \n-void block_iter_seek_start(struct block_iter *it, const struct reftable_block *block)\n+void block_iter_init(struct block_iter *it, const struct reftable_block *block)\n {\n \tit->block = block;\n+\tblock_iter_seek_start(it);\n+}\n+\n+void block_iter_seek_start(struct block_iter *it)\n+{\n \treftable_buf_reset(&it->last_key);\n-\tit->next_off = block->header_off + 4;\n+\tit->next_off = it->block->header_off + 4;\n }\n \n struct restart_needle_less_args {\n@@ -473,12 +478,11 @@ void block_iter_close(struct block_iter *it)\n \treftable_buf_release(&it->scratch);\n }\n \n-int block_iter_seek_key(struct block_iter *it, const struct reftable_block *block,\n-\t\t\tstruct reftable_buf *want)\n+int block_iter_seek_key(struct block_iter *it, struct reftable_buf *want)\n {\n \tstruct restart_needle_less_args args = {\n \t\t.needle = *want,\n-\t\t.block = block,\n+\t\t.block = it->block,\n \t};\n \tstruct reftable_record rec;\n \tint err = 0;\n@@ -496,7 +500,7 @@ int block_iter_seek_key(struct block_iter *it, const struct reftable_block *bloc\n \t * restart point. While that works alright, we would end up scanning\n \t * too many record.\n \t */\n-\ti = binsearch(block->restart_count, &restart_needle_less, &args);\n+\ti = binsearch(it->block->restart_count, &restart_needle_less, &args);\n \tif (args.error) {\n \t\terr = REFTABLE_FORMAT_ERROR;\n \t\tgoto done;\n@@ -521,12 +525,11 @@ int block_iter_seek_key(struct block_iter *it, const struct reftable_block *bloc\n \t *     starting from the preceding restart point.\n \t */\n \tif (i > 0)\n-\t\tit->next_off = block_restart_offset(block, i - 1);\n+\t\tit->next_off = block_restart_offset(it->block, i - 1);\n \telse\n-\t\tit->next_off = block->header_off + 4;\n-\tit->block = block;\n+\t\tit->next_off = it->block->header_off + 4;\n \n-\terr = reftable_record_init(&rec, reftable_block_type(block));\n+\terr = reftable_record_init(&rec, reftable_block_type(it->block));\n \tif (err < 0)\n \t\tgoto done;\n \ndiff --git a/reftable/block.h b/reftable/block.h\nindex 268d5a1e005..1bfd44f56aa 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -79,12 +79,23 @@ struct block_iter {\n \t.scratch = REFTABLE_BUF_INIT, \\\n }\n \n-/* Position `it` at start of the block */\n-void block_iter_seek_start(struct block_iter *it, const struct reftable_block *block);\n+/*\n+ * Initialize the block iterator with the given block. The iterator will be\n+ * positioned at the first record contained in the block. The block must remain\n+ * valid until the end of the iterator's lifetime. It is valid to re-initialize\n+ * iterators multiple times.\n+ */\n+void block_iter_init(struct block_iter *it, const struct reftable_block *block);\n+\n+/* Position the initialized iterator at the first record of its block. */\n+void block_iter_seek_start(struct block_iter *it);\n \n-/* Position `it` to the `want` key in the block */\n-int block_iter_seek_key(struct block_iter *it, const struct reftable_block *block,\n-\t\t\tstruct reftable_buf *want);\n+/*\n+ * Position the initialized iterator at the desired record key. It is not an\n+ * error in case the record cannot be found. If so, a subsequent call to\n+ * `block_iter_next()` will indicate that the iterator is exhausted.\n+ */\n+int block_iter_seek_key(struct block_iter *it, struct reftable_buf *want);\n \n /* return < 0 for error, 0 for OK, > 0 for EOF. */\n int block_iter_next(struct block_iter *it, struct reftable_record *rec);\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex 74684de8f6a..7a7e8aa4d7e 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -139,7 +139,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n \t\t/* indexed block does not exist. */\n \t\treturn REFTABLE_FORMAT_ERROR;\n \t}\n-\tblock_iter_seek_start(&it->cur, &it->block);\n+\tblock_iter_init(&it->cur, &it->block);\n \treturn 0;\n }\n \ndiff --git a/reftable/table.c b/reftable/table.c\nindex 3f2e70caac4..5422ed6769c 100644\n--- a/reftable/table.c\n+++ b/reftable/table.c\n@@ -208,7 +208,7 @@ static int table_iter_next_block(struct table_iter *ti)\n \n \tti->block_off = next_block_off;\n \tti->is_finished = 0;\n-\tblock_iter_seek_start(&ti->bi, &ti->block);\n+\tblock_iter_init(&ti->bi, &ti->block);\n \n \treturn 0;\n }\n@@ -256,7 +256,7 @@ static int table_iter_seek_to(struct table_iter *ti, uint64_t off, uint8_t typ)\n \n \tti->typ = reftable_block_type(&ti->block);\n \tti->block_off = off;\n-\tblock_iter_seek_start(&ti->bi, &ti->block);\n+\tblock_iter_init(&ti->bi, &ti->block);\n \tti->is_finished = 0;\n \treturn 0;\n }\n@@ -349,7 +349,8 @@ static int table_iter_seek_linear(struct table_iter *ti,\n \t * the wanted key inside of it. If the block does not contain our key\n \t * we know that the corresponding record does not exist.\n \t */\n-\terr = block_iter_seek_key(&ti->bi, &ti->block, &want_key);\n+\tblock_iter_init(&ti->bi, &ti->block);\n+\terr = block_iter_seek_key(&ti->bi, &want_key);\n \tif (err < 0)\n \t\tgoto done;\n \terr = 0;\n@@ -417,7 +418,9 @@ static int table_iter_seek_indexed(struct table_iter *ti,\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = block_iter_seek_key(&ti->bi, &ti->block, &want_index.u.idx.last_key);\n+\t\tblock_iter_init(&ti->bi, &ti->block);\n+\n+\t\terr = block_iter_seek_key(&ti->bi, &want_index.u.idx.last_key);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \ndiff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c\nindex e36ed7ac576..c4ced39a73b 100644\n--- a/t/unit-tests/t-reftable-block.c\n+++ b/t/unit-tests/t-reftable-block.c\n@@ -66,7 +66,8 @@ static void t_ref_block_read_write(void)\n \tblock_source_from_buf(&source ,&block_data);\n \treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n-\tblock_iter_seek_start(&it, &block);\n+\tblock_iter_init(&it, &block);\n+\tblock_iter_seek_start(&it);\n \n \tfor (i = 0; ; i++) {\n \t\tret = block_iter_next(&it, &rec);\n@@ -79,10 +80,9 @@ static void t_ref_block_read_write(void)\n \t}\n \n \tfor (i = 0; i < N; i++) {\n-\t\tblock_iter_reset(&it);\n \t\treftable_record_key(&recs[i], &want);\n \n-\t\tret = block_iter_seek_key(&it, &block, &want);\n+\t\tret = block_iter_seek_key(&it, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -91,7 +91,7 @@ static void t_ref_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1));\n \n \t\twant.len--;\n-\t\tret = block_iter_seek_key(&it, &block, &want);\n+\t\tret = block_iter_seek_key(&it, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -156,7 +156,8 @@ static void t_log_block_read_write(void)\n \tblock_source_from_buf(&source, &block_data);\n \treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n-\tblock_iter_seek_start(&it, &block);\n+\tblock_iter_init(&it, &block);\n+\tblock_iter_seek_start(&it);\n \n \tfor (i = 0; ; i++) {\n \t\tret = block_iter_next(&it, &rec);\n@@ -169,11 +170,10 @@ static void t_log_block_read_write(void)\n \t}\n \n \tfor (i = 0; i < N; i++) {\n-\t\tblock_iter_reset(&it);\n \t\treftable_buf_reset(&want);\n \t\tcheck(!reftable_buf_addstr(&want, recs[i].u.log.refname));\n \n-\t\tret = block_iter_seek_key(&it, &block, &want);\n+\t\tret = block_iter_seek_key(&it, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -182,7 +182,7 @@ static void t_log_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1));\n \n \t\twant.len--;\n-\t\tret = block_iter_seek_key(&it, &block, &want);\n+\t\tret = block_iter_seek_key(&it, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -249,7 +249,8 @@ static void t_obj_block_read_write(void)\n \tblock_source_from_buf(&source, &block_data);\n \treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n-\tblock_iter_seek_start(&it, &block);\n+\tblock_iter_init(&it, &block);\n+\tblock_iter_seek_start(&it);\n \n \tfor (i = 0; ; i++) {\n \t\tret = block_iter_next(&it, &rec);\n@@ -262,10 +263,9 @@ static void t_obj_block_read_write(void)\n \t}\n \n \tfor (i = 0; i < N; i++) {\n-\t\tblock_iter_reset(&it);\n \t\treftable_record_key(&recs[i], &want);\n \n-\t\tret = block_iter_seek_key(&it, &block, &want);\n+\t\tret = block_iter_seek_key(&it, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -334,7 +334,8 @@ static void t_index_block_read_write(void)\n \tblock_source_from_buf(&source, &block_data);\n \treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n-\tblock_iter_seek_start(&it, &block);\n+\tblock_iter_init(&it, &block);\n+\tblock_iter_seek_start(&it);\n \n \tfor (i = 0; ; i++) {\n \t\tret = block_iter_next(&it, &rec);\n@@ -347,10 +348,9 @@ static void t_index_block_read_write(void)\n \t}\n \n \tfor (i = 0; i < N; i++) {\n-\t\tblock_iter_reset(&it);\n \t\treftable_record_key(&recs[i], &want);\n \n-\t\tret = block_iter_seek_key(&it, &block, &want);\n+\t\tret = block_iter_seek_key(&it, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -359,7 +359,7 @@ static void t_index_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1));\n \n \t\twant.len--;\n-\t\tret = block_iter_seek_key(&it, &block, &want);\n+\t\tret = block_iter_seek_key(&it, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515350","messageId":"20250331-pks-reftable-polishing-v1-12-ebed5247434c@pks.im","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im","subject":"[PATCH 12/16] reftable/block: expose a generic iterator over reftable records","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-31T08:41:30Z","receivedAt":"2025-03-31T08:41:34Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Expose a generic iterator over reftable records and expose it via the\npublic interface. Together with an upcoming iterator for reftable blocks\ncontained in a table this will allow users to trivially iterate through\nblocks and their respective records individually.\n\nThis functionality will be used to implement consistency checks for the\nreftable backend, which requires more fine-grained control over how we\nread data.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/block.c                | 56 +++++++++++++++++++++++++++++\n reftable/block.h                |  2 +-\n reftable/reftable-block.h       |  5 +++\n t/unit-tests/t-reftable-block.c | 78 +++++++++++++++++++++++++++++++++++++++++\n 4 files changed, 140 insertions(+), 1 deletion(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 6e5c1191f5f..fb91090079b 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -10,6 +10,7 @@\n \n #include \"blocksource.h\"\n #include \"constants.h\"\n+#include \"iter.h\"\n #include \"record.h\"\n #include \"reftable-error.h\"\n #include \"system.h\"\n@@ -581,6 +582,61 @@ int block_iter_seek_key(struct block_iter *it, struct reftable_buf *want)\n \treturn err;\n }\n \n+static int block_iter_seek_void(void *it, struct reftable_record *want)\n+{\n+\tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n+\tstruct block_iter *bi = it;\n+\tint err;\n+\n+\tif (bi->block->block_type != want->type)\n+\t\treturn REFTABLE_API_ERROR;\n+\n+\terr = reftable_record_key(want, &buf);\n+\tif (err < 0)\n+\t\tgoto out;\n+\n+\terr = block_iter_seek_key(it, &buf);\n+\tif (err < 0)\n+\t\tgoto out;\n+\n+\terr = 0;\n+\n+out:\n+\treftable_buf_release(&buf);\n+\treturn err;\n+}\n+\n+static int block_iter_next_void(void *it, struct reftable_record *rec)\n+{\n+\treturn block_iter_next(it, rec);\n+}\n+\n+static void block_iter_close_void(void *it)\n+{\n+\tblock_iter_close(it);\n+}\n+\n+static struct reftable_iterator_vtable block_iter_vtable = {\n+\t.seek = &block_iter_seek_void,\n+\t.next = &block_iter_next_void,\n+\t.close = &block_iter_close_void,\n+};\n+\n+int reftable_block_init_iterator(const struct reftable_block *b,\n+\t\t\t\t struct reftable_iterator *it)\n+{\n+\tstruct block_iter *bi;\n+\n+\tREFTABLE_CALLOC_ARRAY(bi, 1);\n+\tblock_iter_init(bi, b);\n+\n+\tassert(!it->ops);\n+\tit->iter_arg = bi;\n+\tit->ops = &block_iter_vtable;\n+\n+\treturn 0;\n+}\n+\n void block_writer_release(struct block_writer *bw)\n {\n \tdeflateEnd(bw->zstream);\ndiff --git a/reftable/block.h b/reftable/block.h\nindex 1bfd44f56aa..3d069393922 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -63,7 +63,7 @@ int block_writer_finish(struct block_writer *w);\n /* clears out internally allocated block_writer members. */\n void block_writer_release(struct block_writer *bw);\n \n-/* Iterate over entries in a block */\n+/* Iterator for records contained in a single block. */\n struct block_iter {\n \t/* offset within the block of the next entry to read. */\n \tuint32_t next_off;\ndiff --git a/reftable/reftable-block.h b/reftable/reftable-block.h\nindex ce1d9d3ce79..ca13e5dd9c3 100644\n--- a/reftable/reftable-block.h\n+++ b/reftable/reftable-block.h\n@@ -13,6 +13,7 @@\n \n #include \"reftable-basics.h\"\n #include \"reftable-blocksource.h\"\n+#include \"reftable-iterator.h\"\n \n struct z_stream_s;\n \n@@ -60,6 +61,10 @@ int reftable_block_init(struct reftable_block *b,\n /* Release resources allocated by the block. */\n void reftable_block_release(struct reftable_block *b);\n \n+/* Initialize a generic record iterator from the given block. */\n+int reftable_block_init_iterator(const struct reftable_block *b,\n+\t\t\t\t struct reftable_iterator *it);\n+\n /* Returns the block type (eg. 'r' for refs). */\n uint8_t reftable_block_type(const struct reftable_block *b);\n \ndiff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c\nindex c4ced39a73b..e092d0bb8f8 100644\n--- a/t/unit-tests/t-reftable-block.c\n+++ b/t/unit-tests/t-reftable-block.c\n@@ -376,12 +376,90 @@ static void t_index_block_read_write(void)\n \t\treftable_record_release(&recs[i]);\n }\n \n+static void t_block_iterator(void)\n+{\n+\tstruct reftable_block_source source = { 0 };\n+\tstruct block_writer writer = {\n+\t\t.last_key = REFTABLE_BUF_INIT,\n+\t};\n+\tstruct reftable_record expected_refs[20];\n+\tstruct reftable_ref_record ref = { 0 };\n+\tstruct reftable_iterator it = { 0 };\n+\tstruct reftable_block block = { 0 };\n+\tstruct reftable_buf data;\n+\tint err;\n+\n+\tdata.len = 1024;\n+\tREFTABLE_CALLOC_ARRAY(data.buf, data.len);\n+\tcheck(data.buf != NULL);\n+\n+\terr = block_writer_init(&writer, BLOCK_TYPE_REF, (uint8_t *) data.buf, data.len,\n+\t\t\t\t0, hash_size(REFTABLE_HASH_SHA1));\n+\tcheck(!err);\n+\n+\tfor (size_t i = 0; i < ARRAY_SIZE(expected_refs); i++) {\n+\t\texpected_refs[i] = (struct reftable_record) {\n+\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t.u.ref = {\n+\t\t\t\t.value_type = REFTABLE_REF_VAL1,\n+\t\t\t\t.refname = xstrfmt(\"refs/heads/branch-%02\"PRIuMAX, (uintmax_t)i),\n+\t\t\t},\n+\t\t};\n+\t\tmemset(expected_refs[i].u.ref.value.val1, i, REFTABLE_HASH_SIZE_SHA1);\n+\n+\t\terr = block_writer_add(&writer, &expected_refs[i]);\n+\t\tcheck_int(err, ==, 0);\n+\t}\n+\n+\terr = block_writer_finish(&writer);\n+\tcheck_int(err, >, 0);\n+\n+\tblock_source_from_buf(&source, &data);\n+\treftable_block_init(&block, &source, 0, 0, data.len, REFTABLE_HASH_SIZE_SHA1);\n+\n+\terr = reftable_block_init_iterator(&block, &it);\n+\tcheck_int(err, ==, 0);\n+\n+\tfor (size_t i = 0; ; i++) {\n+\t\terr = reftable_iterator_next_ref(&it, &ref);\n+\t\tif (err > 0) {\n+\t\t\tcheck_int(i, ==, ARRAY_SIZE(expected_refs));\n+\t\t\tbreak;\n+\t\t}\n+\t\tcheck_int(err, ==, 0);\n+\n+\t\tcheck(reftable_ref_record_equal(&ref, &expected_refs[i].u.ref,\n+\t\t\t\t\t\tREFTABLE_HASH_SIZE_SHA1));\n+\t}\n+\n+\terr = reftable_iterator_seek_ref(&it, \"refs/heads/does-not-exist\");\n+\tcheck_int(err, ==, 0);\n+\terr = reftable_iterator_next_ref(&it, &ref);\n+\tcheck_int(err, ==, 1);\n+\n+\terr = reftable_iterator_seek_ref(&it, \"refs/heads/branch-13\");\n+\tcheck_int(err, ==, 0);\n+\terr = reftable_iterator_next_ref(&it, &ref);\n+\tcheck_int(err, ==, 0);\n+\tcheck(reftable_ref_record_equal(&ref, &expected_refs[13].u.ref,\n+\t\t\t\t\tREFTABLE_HASH_SIZE_SHA1));\n+\n+\tfor (size_t i = 0; i < ARRAY_SIZE(expected_refs); i++)\n+\t\treftable_free(expected_refs[i].u.ref.refname);\n+\treftable_ref_record_release(&ref);\n+\treftable_iterator_destroy(&it);\n+\treftable_block_release(&block);\n+\tblock_writer_release(&writer);\n+\treftable_buf_release(&data);\n+}\n+\n int cmd_main(int argc UNUSED, const char *argv[] UNUSED)\n {\n \tTEST(t_index_block_read_write(), \"read-write operations on index blocks work\");\n \tTEST(t_log_block_read_write(), \"read-write operations on log blocks work\");\n \tTEST(t_obj_block_read_write(), \"read-write operations on obj blocks work\");\n \tTEST(t_ref_block_read_write(), \"read-write operations on ref blocks work\");\n+\tTEST(t_block_iterator(), \"block iterator works\");\n \n \treturn test_done();\n }\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515351","messageId":"20250331-pks-reftable-polishing-v1-13-ebed5247434c@pks.im","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im","subject":"[PATCH 13/16] reftable/table: add `reftable_table` to the public interface","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-31T08:41:31Z","receivedAt":"2025-03-31T08:41:35Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The `reftable_table` interface is an internal implementation detail that\ncallers have no access to. Having direct access to this structure is\nimportant though for a subsequent patch series that will implement\nconsistency checks for the reftable backend.\n\nMove the structure into \"reftable-table.h\" so that it part of the public\ninterface.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/reftable-table.h | 32 +++++++++++++++++++++++++++++++-\n reftable/table.h          | 33 ---------------------------------\n 2 files changed, 31 insertions(+), 34 deletions(-)\n\ndiff --git a/reftable/reftable-table.h b/reftable/reftable-table.h\nindex 94379026727..a78db9eea7e 100644\n--- a/reftable/reftable-table.h\n+++ b/reftable/reftable-table.h\n@@ -20,8 +20,38 @@\n  * reftable_merged_table and struct reftable_stack.\n  */\n \n+/* Metadata for a block type. */\n+struct reftable_table_offsets {\n+\tint is_present;\n+\tuint64_t offset;\n+\tuint64_t index_offset;\n+};\n+\n /* The table struct is a handle to an open reftable file. */\n-struct reftable_table;\n+struct reftable_table {\n+\t/* for convenience, associate a name with the instance. */\n+\tchar *name;\n+\tstruct reftable_block_source source;\n+\n+\t/* Size of the file, excluding the footer. */\n+\tuint64_t size;\n+\n+\t/* The hash function used for ref records. */\n+\tenum reftable_hash hash_id;\n+\n+\tuint32_t block_size;\n+\tuint64_t min_update_index;\n+\tuint64_t max_update_index;\n+\t/* Length of the OID keys in the 'o' section */\n+\tint object_id_len;\n+\tint version;\n+\n+\tstruct reftable_table_offsets ref_offsets;\n+\tstruct reftable_table_offsets obj_offsets;\n+\tstruct reftable_table_offsets log_offsets;\n+\n+\tuint64_t refcount;\n+};\n \n /* reftable_table_new opens a reftable for reading. If successful,\n  * returns 0 code and sets pp. The name is used for creating a\ndiff --git a/reftable/table.h b/reftable/table.h\nindex e15d58d8e9e..c54703e6210 100644\n--- a/reftable/table.h\n+++ b/reftable/table.h\n@@ -14,39 +14,6 @@\n #include \"reftable-iterator.h\"\n #include \"reftable-table.h\"\n \n-/* metadata for a block type */\n-struct reftable_table_offsets {\n-\tint is_present;\n-\tuint64_t offset;\n-\tuint64_t index_offset;\n-};\n-\n-/* The state for reading a reftable file. */\n-struct reftable_table {\n-\t/* for convenience, associate a name with the instance. */\n-\tchar *name;\n-\tstruct reftable_block_source source;\n-\n-\t/* Size of the file, excluding the footer. */\n-\tuint64_t size;\n-\n-\t/* The hash function used for ref records. */\n-\tenum reftable_hash hash_id;\n-\n-\tuint32_t block_size;\n-\tuint64_t min_update_index;\n-\tuint64_t max_update_index;\n-\t/* Length of the OID keys in the 'o' section */\n-\tint object_id_len;\n-\tint version;\n-\n-\tstruct reftable_table_offsets ref_offsets;\n-\tstruct reftable_table_offsets obj_offsets;\n-\tstruct reftable_table_offsets log_offsets;\n-\n-\tuint64_t refcount;\n-};\n-\n const char *reftable_table_name(struct reftable_table *t);\n \n int table_init_iter(struct reftable_table *t,\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515352","messageId":"20250331-pks-reftable-polishing-v1-14-ebed5247434c@pks.im","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im","subject":"[PATCH 14/16] reftable/table: introduce iterator for table blocks","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-31T08:41:32Z","receivedAt":"2025-03-31T08:41:35Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new iterator that allows the caller to iterate through all\nblocks contained in a table. This gives users more fine-grained control\nover how exactly those blocks are being read and exposes information to\ncallers that was previously inaccessible.\n\nThis iterator will be required by a future patch series that adds\nconsistency checks for the reftable backend. In addition to that though\nwe will also reimplement `reftable_table_print_blocks()` on top of this\nnew iterator in a subsequent commit.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/reftable-table.h       |  16 ++++++\n reftable/table.c                |  47 +++++++++++++++++\n t/unit-tests/t-reftable-table.c | 109 ++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 172 insertions(+)\n\ndiff --git a/reftable/reftable-table.h b/reftable/reftable-table.h\nindex a78db9eea7e..f0f1784c664 100644\n--- a/reftable/reftable-table.h\n+++ b/reftable/reftable-table.h\n@@ -10,6 +10,7 @@\n #define REFTABLE_TABLE_H\n \n #include \"reftable-iterator.h\"\n+#include \"reftable-block.h\"\n #include \"reftable-blocksource.h\"\n \n /*\n@@ -99,4 +100,19 @@ uint64_t reftable_table_min_update_index(struct reftable_table *t);\n /* print blocks onto stdout for debugging. */\n int reftable_table_print_blocks(const char *tablename);\n \n+/*\n+ * An iterator that iterates through the blocks contained in a given table.\n+ */\n+struct reftable_table_iterator {\n+\tvoid *iter_arg;\n+};\n+\n+int reftable_table_init_table_iterator(struct reftable_table *t,\n+\t\t\t\t       struct reftable_table_iterator *it);\n+\n+void reftable_table_iterator_release(struct reftable_table_iterator *it);\n+\n+int reftable_table_iterator_next(struct reftable_table_iterator *it,\n+\t\t\t\t const struct reftable_block **out);\n+\n #endif\ndiff --git a/reftable/table.c b/reftable/table.c\nindex 5422ed6769c..d84a87e7ad0 100644\n--- a/reftable/table.c\n+++ b/reftable/table.c\n@@ -804,3 +804,50 @@ int reftable_table_print_blocks(const char *tablename)\n \ttable_iter_close(&ti);\n \treturn err;\n }\n+\n+int reftable_table_init_table_iterator(struct reftable_table *t,\n+\t\t\t\t       struct reftable_table_iterator *it)\n+{\n+\tstruct table_iter *ti;\n+\tint err;\n+\n+\tREFTABLE_ALLOC_ARRAY(ti, 1);\n+\tif (!ti)\n+\t\treturn REFTABLE_OUT_OF_MEMORY_ERROR;\n+\n+\terr = table_iter_init(ti, t);\n+\tif (err < 0)\n+\t\tgoto out;\n+\n+\tit->iter_arg = ti;\n+\terr = 0;\n+\n+out:\n+\tif (err < 0)\n+\t\treftable_free(ti);\n+\treturn err;\n+}\n+\n+void reftable_table_iterator_release(struct reftable_table_iterator *it)\n+{\n+\tif (!it->iter_arg)\n+\t\treturn;\n+\ttable_iter_close(it->iter_arg);\n+\treftable_free(it->iter_arg);\n+\tit->iter_arg = NULL;\n+}\n+\n+int reftable_table_iterator_next(struct reftable_table_iterator *it,\n+\t\t\t\t const struct reftable_block **out)\n+{\n+\tstruct table_iter *ti = it->iter_arg;\n+\tint err;\n+\n+\terr = table_iter_next_block(ti);\n+\tif (err)\n+\t\treturn err;\n+\n+\t*out = &ti->block;\n+\n+\treturn 0;\n+}\ndiff --git a/t/unit-tests/t-reftable-table.c b/t/unit-tests/t-reftable-table.c\nindex 77c59dbf46d..58b13ad496f 100644\n--- a/t/unit-tests/t-reftable-table.c\n+++ b/t/unit-tests/t-reftable-table.c\n@@ -1,7 +1,10 @@\n #include \"test-lib.h\"\n #include \"lib-reftable.h\"\n #include \"reftable/blocksource.h\"\n+#include \"reftable/constants.h\"\n+#include \"reftable/iter.h\"\n #include \"reftable/table.h\"\n+#include \"strbuf.h\"\n \n static int t_table_seek_once(void)\n {\n@@ -88,9 +91,115 @@ static int t_table_reseek(void)\n \treturn 0;\n }\n \n+static int t_table_block_iterator(void)\n+{\n+\tstruct reftable_block_source source = { 0 };\n+\tstruct reftable_table_iterator it = { 0 };\n+\tstruct reftable_ref_record *records;\n+\tconst struct reftable_block *block;\n+\tstruct reftable_table *table;\n+\tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n+\tstruct {\n+\t\tuint8_t block_type;\n+\t\tuint16_t header_off;\n+\t\tuint16_t restart_count;\n+\t\tuint16_t record_count;\n+\t} expected_blocks[] = {\n+\t\t{\n+\t\t\t.block_type = BLOCK_TYPE_REF,\n+\t\t\t.header_off = 24,\n+\t\t\t.restart_count = 10,\n+\t\t\t.record_count = 158,\n+\t\t},\n+\t\t{\n+\t\t\t.block_type = BLOCK_TYPE_REF,\n+\t\t\t.restart_count = 10,\n+\t\t\t.record_count = 159,\n+\t\t},\n+\t\t{\n+\t\t\t.block_type = BLOCK_TYPE_REF,\n+\t\t\t.restart_count = 10,\n+\t\t\t.record_count = 159,\n+\t\t},\n+\t\t{\n+\t\t\t.block_type = BLOCK_TYPE_REF,\n+\t\t\t.restart_count = 2,\n+\t\t\t.record_count = 24,\n+\t\t},\n+\t\t{\n+\t\t\t.block_type = BLOCK_TYPE_INDEX,\n+\t\t\t.restart_count = 1,\n+\t\t\t.record_count = 4,\n+\t\t},\n+\t\t{\n+\t\t\t.block_type = BLOCK_TYPE_OBJ,\n+\t\t\t.restart_count = 1,\n+\t\t\t.record_count = 1,\n+\t\t},\n+\t};\n+\tconst size_t nrecords = 500;\n+\tint ret;\n+\n+\tREFTABLE_CALLOC_ARRAY(records, nrecords);\n+\tfor (size_t i = 0; i < nrecords; i++) {\n+\t\trecords[i].value_type = REFTABLE_REF_VAL1,\n+\t\trecords[i].refname = xstrfmt(\"refs/heads/branch-%03\"PRIuMAX, (uintmax_t) i);\n+\t}\n+\n+\tt_reftable_write_to_buf(&buf, records, nrecords, NULL, 0, NULL);\n+\tblock_source_from_buf(&source, &buf);\n+\n+\tret = reftable_table_new(&table, &source, \"name\");\n+\tcheck(!ret);\n+\n+\tret = reftable_table_init_table_iterator(table, &it);\n+\tcheck(!ret);\n+\n+\tfor (size_t i = 0; i < ARRAY_SIZE(expected_blocks); i++) {\n+\t\tstruct reftable_iterator record_it = { 0 };\n+\t\tstruct reftable_record record = {\n+\t\t\t.type = expected_blocks[i].block_type,\n+\t\t};\n+\n+\t\tret = reftable_table_iterator_next(&it, &block);\n+\t\tcheck(!ret);\n+\n+\t\tcheck_int(block->block_type, ==, expected_blocks[i].block_type);\n+\t\tcheck_int(block->header_off, ==, expected_blocks[i].header_off);\n+\t\tcheck_int(block->restart_count, ==, expected_blocks[i].restart_count);\n+\n+\t\tret = reftable_block_init_iterator(block, &record_it);\n+\t\tcheck(!ret);\n+\n+\t\tfor (size_t j = 0; ; j++) {\n+\t\t\tret = iterator_next(&record_it, &record);\n+\t\t\tif (ret > 0) {\n+\t\t\t\tcheck_int(j, ==, expected_blocks[i].record_count);\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t\tcheck(!ret);\n+\t\t}\n+\n+\t\treftable_iterator_destroy(&record_it);\n+\t\treftable_record_release(&record);\n+\t}\n+\n+\tret = reftable_table_iterator_next(&it, &block);\n+\tcheck_int(ret, ==, 1);\n+\n+\tfor (size_t i = 0; i < nrecords; i++)\n+\t\treftable_free(records[i].refname);\n+\treftable_table_iterator_release(&it);\n+\treftable_table_decref(table);\n+\treftable_buf_release(&buf);\n+\treftable_free(records);\n+\treturn 0;\n+}\n+\n int cmd_main(int argc UNUSED, const char *argv[] UNUSED)\n {\n \tTEST(t_table_seek_once(), \"table can seek once\");\n \tTEST(t_table_reseek(), \"table can reseek multiple times\");\n+\tTEST(t_table_block_iterator(), \"table can iterate through blocks\");\n \treturn test_done();\n }\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515354","messageId":"20250331-pks-reftable-polishing-v1-15-ebed5247434c@pks.im","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im","subject":"[PATCH 15/16] reftable/constants: make block types part of the public interface","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-31T08:41:33Z","receivedAt":"2025-03-31T08:41:36Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Now that reftable blocks can be read individually via the public\ninterface it becomes necessary for callers to be able to distinguish the\ndifferent types of blocks. Expose the relevant constants.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/block.c                 |  4 ++--\n reftable/constants.h             |  6 +-----\n reftable/iter.c                  | 10 +++++-----\n reftable/merged.c                |  4 ++--\n reftable/record.c                | 40 ++++++++++++++++++++--------------------\n reftable/reftable-constants.h    | 18 ++++++++++++++++++\n reftable/stack.c                 |  8 ++++----\n reftable/table.c                 | 40 ++++++++++++++++++++--------------------\n reftable/writer.c                | 22 +++++++++++-----------\n t/unit-tests/t-reftable-block.c  | 22 +++++++++++-----------\n t/unit-tests/t-reftable-merged.c | 12 ++++++------\n t/unit-tests/t-reftable-pq.c     | 10 +++++-----\n t/unit-tests/t-reftable-record.c | 40 ++++++++++++++++++++--------------------\n t/unit-tests/t-reftable-table.c  | 12 ++++++------\n 14 files changed, 131 insertions(+), 117 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex fb91090079b..393da2026ce 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -160,7 +160,7 @@ int block_writer_finish(struct block_writer *w)\n \t * Log records are stored zlib-compressed. Note that the compression\n \t * also spans over the restart points we have just written.\n \t */\n-\tif (block_writer_type(w) == BLOCK_TYPE_LOG) {\n+\tif (block_writer_type(w) == REFTABLE_BLOCK_TYPE_LOG) {\n \t\tint block_header_skip = 4 + w->header_off;\n \t\tuLongf src_len = w->next - block_header_skip, compressed_len;\n \t\tint ret;\n@@ -254,7 +254,7 @@ int reftable_block_init(struct reftable_block *block,\n \t\t\tgoto done;\n \t}\n \n-\tif (block_type == BLOCK_TYPE_LOG) {\n+\tif (block_type == REFTABLE_BLOCK_TYPE_LOG) {\n \t\tuint32_t block_header_skip = 4 + header_size;\n \t\tuLong dst_len = block_size - block_header_skip;\n \t\tuLong src_len = block->block.len - block_header_skip;\ndiff --git a/reftable/constants.h b/reftable/constants.h\nindex 091728cf033..e3b1aaa5164 100644\n--- a/reftable/constants.h\n+++ b/reftable/constants.h\n@@ -9,11 +9,7 @@\n #ifndef CONSTANTS_H\n #define CONSTANTS_H\n \n-#define BLOCK_TYPE_LOG 'g'\n-#define BLOCK_TYPE_INDEX 'i'\n-#define BLOCK_TYPE_REF 'r'\n-#define BLOCK_TYPE_OBJ 'o'\n-#define BLOCK_TYPE_ANY 0\n+#include \"reftable-constants.h\"\n \n #define MAX_RESTARTS ((1 << 16) - 1)\n #define DEFAULT_BLOCK_SIZE 4096\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex 7a7e8aa4d7e..7765494d068 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -131,7 +131,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n \tblock_source_release_data(&it->block.block);\n \n \toff = it->offsets[it->offset_idx++];\n-\terr = table_init_block(it->table, &it->block, off, BLOCK_TYPE_REF);\n+\terr = table_init_block(it->table, &it->block, off, REFTABLE_BLOCK_TYPE_REF);\n \tif (err < 0) {\n \t\treturn err;\n \t}\n@@ -246,7 +246,7 @@ int reftable_iterator_seek_ref(struct reftable_iterator *it,\n \t\t\t       const char *name)\n {\n \tstruct reftable_record want = {\n-\t\t.type = BLOCK_TYPE_REF,\n+\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t\t.u.ref = {\n \t\t\t.refname = (char *)name,\n \t\t},\n@@ -258,7 +258,7 @@ int reftable_iterator_next_ref(struct reftable_iterator *it,\n \t\t\t       struct reftable_ref_record *ref)\n {\n \tstruct reftable_record rec = {\n-\t\t.type = BLOCK_TYPE_REF,\n+\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t\t.u = {\n \t\t\t.ref = *ref\n \t\t},\n@@ -272,7 +272,7 @@ int reftable_iterator_seek_log_at(struct reftable_iterator *it,\n \t\t\t\t  const char *name, uint64_t update_index)\n {\n \tstruct reftable_record want = {\n-\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t\t.u.log = {\n \t\t\t.refname = (char *)name,\n \t\t\t.update_index = update_index,\n@@ -291,7 +291,7 @@ int reftable_iterator_next_log(struct reftable_iterator *it,\n \t\t\t       struct reftable_log_record *log)\n {\n \tstruct reftable_record rec = {\n-\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t\t.u = {\n \t\t\t.log = *log,\n \t\t},\ndiff --git a/reftable/merged.c b/reftable/merged.c\nindex d5b974d660e..733de07454d 100644\n--- a/reftable/merged.c\n+++ b/reftable/merged.c\n@@ -301,13 +301,13 @@ int merged_table_init_iter(struct reftable_merged_table *mt,\n int reftable_merged_table_init_ref_iterator(struct reftable_merged_table *mt,\n \t\t\t\t\t    struct reftable_iterator *it)\n {\n-\treturn merged_table_init_iter(mt, it, BLOCK_TYPE_REF);\n+\treturn merged_table_init_iter(mt, it, REFTABLE_BLOCK_TYPE_REF);\n }\n \n int reftable_merged_table_init_log_iterator(struct reftable_merged_table *mt,\n \t\t\t\t\t    struct reftable_iterator *it)\n {\n-\treturn merged_table_init_iter(mt, it, BLOCK_TYPE_LOG);\n+\treturn merged_table_init_iter(mt, it, REFTABLE_BLOCK_TYPE_LOG);\n }\n \n enum reftable_hash reftable_merged_table_hash_id(struct reftable_merged_table *mt)\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 26cd834d405..ed00a724417 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -69,10 +69,10 @@ int put_var_int(struct string_view *dest, uint64_t value)\n int reftable_is_block_type(uint8_t typ)\n {\n \tswitch (typ) {\n-\tcase BLOCK_TYPE_REF:\n-\tcase BLOCK_TYPE_LOG:\n-\tcase BLOCK_TYPE_OBJ:\n-\tcase BLOCK_TYPE_INDEX:\n+\tcase REFTABLE_BLOCK_TYPE_REF:\n+\tcase REFTABLE_BLOCK_TYPE_LOG:\n+\tcase REFTABLE_BLOCK_TYPE_OBJ:\n+\tcase REFTABLE_BLOCK_TYPE_INDEX:\n \t\treturn 1;\n \t}\n \treturn 0;\n@@ -462,7 +462,7 @@ static int reftable_ref_record_cmp_void(const void *_a, const void *_b)\n \n static struct reftable_record_vtable reftable_ref_record_vtable = {\n \t.key = &reftable_ref_record_key,\n-\t.type = BLOCK_TYPE_REF,\n+\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t.copy_from = &reftable_ref_record_copy_from,\n \t.val_type = &reftable_ref_record_val_type,\n \t.encode = &reftable_ref_record_encode,\n@@ -664,7 +664,7 @@ static int reftable_obj_record_cmp_void(const void *_a, const void *_b)\n \n static struct reftable_record_vtable reftable_obj_record_vtable = {\n \t.key = &reftable_obj_record_key,\n-\t.type = BLOCK_TYPE_OBJ,\n+\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n \t.copy_from = &reftable_obj_record_copy_from,\n \t.val_type = &reftable_obj_record_val_type,\n \t.encode = &reftable_obj_record_encode,\n@@ -1035,7 +1035,7 @@ static int reftable_log_record_is_deletion_void(const void *p)\n \n static struct reftable_record_vtable reftable_log_record_vtable = {\n \t.key = &reftable_log_record_key,\n-\t.type = BLOCK_TYPE_LOG,\n+\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t.copy_from = &reftable_log_record_copy_from,\n \t.val_type = &reftable_log_record_val_type,\n \t.encode = &reftable_log_record_encode,\n@@ -1137,7 +1137,7 @@ static int reftable_index_record_cmp(const void *_a, const void *_b)\n \n static struct reftable_record_vtable reftable_index_record_vtable = {\n \t.key = &reftable_index_record_key,\n-\t.type = BLOCK_TYPE_INDEX,\n+\t.type = REFTABLE_BLOCK_TYPE_INDEX,\n \t.copy_from = &reftable_index_record_copy_from,\n \t.val_type = &reftable_index_record_val_type,\n \t.encode = &reftable_index_record_encode,\n@@ -1280,13 +1280,13 @@ int reftable_log_record_is_deletion(const struct reftable_log_record *log)\n static void *reftable_record_data(struct reftable_record *rec)\n {\n \tswitch (rec->type) {\n-\tcase BLOCK_TYPE_REF:\n+\tcase REFTABLE_BLOCK_TYPE_REF:\n \t\treturn &rec->u.ref;\n-\tcase BLOCK_TYPE_LOG:\n+\tcase REFTABLE_BLOCK_TYPE_LOG:\n \t\treturn &rec->u.log;\n-\tcase BLOCK_TYPE_INDEX:\n+\tcase REFTABLE_BLOCK_TYPE_INDEX:\n \t\treturn &rec->u.idx;\n-\tcase BLOCK_TYPE_OBJ:\n+\tcase REFTABLE_BLOCK_TYPE_OBJ:\n \t\treturn &rec->u.obj;\n \t}\n \tabort();\n@@ -1296,13 +1296,13 @@ static struct reftable_record_vtable *\n reftable_record_vtable(struct reftable_record *rec)\n {\n \tswitch (rec->type) {\n-\tcase BLOCK_TYPE_REF:\n+\tcase REFTABLE_BLOCK_TYPE_REF:\n \t\treturn &reftable_ref_record_vtable;\n-\tcase BLOCK_TYPE_LOG:\n+\tcase REFTABLE_BLOCK_TYPE_LOG:\n \t\treturn &reftable_log_record_vtable;\n-\tcase BLOCK_TYPE_INDEX:\n+\tcase REFTABLE_BLOCK_TYPE_INDEX:\n \t\treturn &reftable_index_record_vtable;\n-\tcase BLOCK_TYPE_OBJ:\n+\tcase REFTABLE_BLOCK_TYPE_OBJ:\n \t\treturn &reftable_obj_record_vtable;\n \t}\n \tabort();\n@@ -1314,11 +1314,11 @@ int reftable_record_init(struct reftable_record *rec, uint8_t typ)\n \trec->type = typ;\n \n \tswitch (typ) {\n-\tcase BLOCK_TYPE_REF:\n-\tcase BLOCK_TYPE_LOG:\n-\tcase BLOCK_TYPE_OBJ:\n+\tcase REFTABLE_BLOCK_TYPE_REF:\n+\tcase REFTABLE_BLOCK_TYPE_LOG:\n+\tcase REFTABLE_BLOCK_TYPE_OBJ:\n \t\treturn 0;\n-\tcase BLOCK_TYPE_INDEX:\n+\tcase REFTABLE_BLOCK_TYPE_INDEX:\n \t\treftable_buf_init(&rec->u.idx.last_key);\n \t\treturn 0;\n \tdefault:\ndiff --git a/reftable/reftable-constants.h b/reftable/reftable-constants.h\nnew file mode 100644\nindex 00000000000..4ae9ba4bacc\n--- /dev/null\n+++ b/reftable/reftable-constants.h\n@@ -0,0 +1,18 @@\n+/*\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n+\n+#ifndef REFTABLE_CONSTANTS_H\n+#define REFTABLE_CONSTANTS_H\n+\n+#define REFTABLE_BLOCK_TYPE_LOG 'g'\n+#define REFTABLE_BLOCK_TYPE_INDEX 'i'\n+#define REFTABLE_BLOCK_TYPE_REF 'r'\n+#define REFTABLE_BLOCK_TYPE_OBJ 'o'\n+#define REFTABLE_BLOCK_TYPE_ANY 0\n+\n+#endif /* REFTABLE_CONSTANTS_H */\ndiff --git a/reftable/stack.c b/reftable/stack.c\nindex cc48e725b14..4caf96aa1d6 100644\n--- a/reftable/stack.c\n+++ b/reftable/stack.c\n@@ -203,14 +203,14 @@ int reftable_stack_init_ref_iterator(struct reftable_stack *st,\n \t\t\t\t      struct reftable_iterator *it)\n {\n \treturn merged_table_init_iter(reftable_stack_merged_table(st),\n-\t\t\t\t      it, BLOCK_TYPE_REF);\n+\t\t\t\t      it, REFTABLE_BLOCK_TYPE_REF);\n }\n \n int reftable_stack_init_log_iterator(struct reftable_stack *st,\n \t\t\t\t     struct reftable_iterator *it)\n {\n \treturn merged_table_init_iter(reftable_stack_merged_table(st),\n-\t\t\t\t      it, BLOCK_TYPE_LOG);\n+\t\t\t\t      it, REFTABLE_BLOCK_TYPE_LOG);\n }\n \n struct reftable_merged_table *\n@@ -1098,7 +1098,7 @@ static int stack_write_compact(struct reftable_stack *st,\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = merged_table_init_iter(mt, &it, BLOCK_TYPE_REF);\n+\terr = merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_REF);\n \tif (err < 0)\n \t\tgoto done;\n \n@@ -1126,7 +1126,7 @@ static int stack_write_compact(struct reftable_stack *st,\n \t}\n \treftable_iterator_destroy(&it);\n \n-\terr = merged_table_init_iter(mt, &it, BLOCK_TYPE_LOG);\n+\terr = merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_LOG);\n \tif (err < 0)\n \t\tgoto done;\n \ndiff --git a/reftable/table.c b/reftable/table.c\nindex d84a87e7ad0..48f0cdfd42b 100644\n--- a/reftable/table.c\n+++ b/reftable/table.c\n@@ -20,11 +20,11 @@ static struct reftable_table_offsets *\n table_offsets_for(struct reftable_table *t, uint8_t typ)\n {\n \tswitch (typ) {\n-\tcase BLOCK_TYPE_REF:\n+\tcase REFTABLE_BLOCK_TYPE_REF:\n \t\treturn &t->ref_offsets;\n-\tcase BLOCK_TYPE_LOG:\n+\tcase REFTABLE_BLOCK_TYPE_LOG:\n \t\treturn &t->log_offsets;\n-\tcase BLOCK_TYPE_OBJ:\n+\tcase REFTABLE_BLOCK_TYPE_OBJ:\n \t\treturn &t->obj_offsets;\n \t}\n \tabort();\n@@ -112,9 +112,9 @@ static int parse_footer(struct reftable_table *t, uint8_t *footer,\n \t}\n \n \tfirst_block_typ = header[header_size(t->version)];\n-\tt->ref_offsets.is_present = (first_block_typ == BLOCK_TYPE_REF);\n+\tt->ref_offsets.is_present = (first_block_typ == REFTABLE_BLOCK_TYPE_REF);\n \tt->ref_offsets.offset = 0;\n-\tt->log_offsets.is_present = (first_block_typ == BLOCK_TYPE_LOG ||\n+\tt->log_offsets.is_present = (first_block_typ == REFTABLE_BLOCK_TYPE_LOG ||\n \t\t\t\t     t->log_offsets.offset > 0);\n \tt->obj_offsets.is_present = t->obj_offsets.offset > 0;\n \tif (t->obj_offsets.is_present && !t->object_id_len) {\n@@ -150,7 +150,7 @@ static int table_iter_next_in_block(struct table_iter *ti,\n \t\t\t\t    struct reftable_record *rec)\n {\n \tint res = block_iter_next(&ti->bi, rec);\n-\tif (res == 0 && reftable_record_type(rec) == BLOCK_TYPE_REF) {\n+\tif (res == 0 && reftable_record_type(rec) == REFTABLE_BLOCK_TYPE_REF) {\n \t\trec->u.ref.update_index += ti->table->min_update_index;\n \t}\n \n@@ -177,7 +177,7 @@ int table_init_block(struct reftable_table *t, struct reftable_block *block,\n \tif (err < 0)\n \t\tgoto done;\n \n-\tif (want_typ != BLOCK_TYPE_ANY && block->block_type != want_typ) {\n+\tif (want_typ != REFTABLE_BLOCK_TYPE_ANY && block->block_type != want_typ) {\n \t\terr = 1;\n \t\tgoto done;\n \t}\n@@ -270,7 +270,7 @@ static int table_iter_seek_start(struct table_iter *ti, uint8_t typ, int index)\n \t\tif (off == 0) {\n \t\t\treturn 1;\n \t\t}\n-\t\ttyp = BLOCK_TYPE_INDEX;\n+\t\ttyp = REFTABLE_BLOCK_TYPE_INDEX;\n \t}\n \n \treturn table_iter_seek_to(ti, off, typ);\n@@ -366,10 +366,10 @@ static int table_iter_seek_indexed(struct table_iter *ti,\n \t\t\t\t   struct reftable_record *rec)\n {\n \tstruct reftable_record want_index = {\n-\t\t.type = BLOCK_TYPE_INDEX, .u.idx = { .last_key = REFTABLE_BUF_INIT }\n+\t\t.type = REFTABLE_BLOCK_TYPE_INDEX, .u.idx = { .last_key = REFTABLE_BUF_INIT }\n \t};\n \tstruct reftable_record index_result = {\n-\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.type = REFTABLE_BLOCK_TYPE_INDEX,\n \t\t.u.idx = { .last_key = REFTABLE_BUF_INIT },\n \t};\n \tint err;\n@@ -429,7 +429,7 @@ static int table_iter_seek_indexed(struct table_iter *ti,\n \t\t\tbreak;\n \t\t}\n \n-\t\tif (ti->typ != BLOCK_TYPE_INDEX) {\n+\t\tif (ti->typ != REFTABLE_BLOCK_TYPE_INDEX) {\n \t\t\terr = REFTABLE_FORMAT_ERROR;\n \t\t\tgoto done;\n \t\t}\n@@ -517,13 +517,13 @@ int table_init_iter(struct reftable_table *t,\n int reftable_table_init_ref_iterator(struct reftable_table *t,\n \t\t\t\t     struct reftable_iterator *it)\n {\n-\treturn table_init_iter(t, it, BLOCK_TYPE_REF);\n+\treturn table_init_iter(t, it, REFTABLE_BLOCK_TYPE_REF);\n }\n \n int reftable_table_init_log_iterator(struct reftable_table *t,\n \t\t\t\t     struct reftable_iterator *it)\n {\n-\treturn table_init_iter(t, it, BLOCK_TYPE_LOG);\n+\treturn table_init_iter(t, it, REFTABLE_BLOCK_TYPE_LOG);\n }\n \n int reftable_table_new(struct reftable_table **out,\n@@ -625,7 +625,7 @@ static int reftable_table_refs_for_indexed(struct reftable_table *t,\n \t\t\t\t\t   uint8_t *oid)\n {\n \tstruct reftable_record want = {\n-\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n \t\t.u.obj = {\n \t\t\t.hash_prefix = oid,\n \t\t\t.hash_prefix_len = t->object_id_len,\n@@ -633,14 +633,14 @@ static int reftable_table_refs_for_indexed(struct reftable_table *t,\n \t};\n \tstruct reftable_iterator oit = { NULL };\n \tstruct reftable_record got = {\n-\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n \t\t.u.obj = { 0 },\n \t};\n \tint err = 0;\n \tstruct indexed_table_ref_iter *itr = NULL;\n \n \t/* Look through the reverse index. */\n-\terr = table_init_iter(t, &oit, BLOCK_TYPE_OBJ);\n+\terr = table_init_iter(t, &oit, REFTABLE_BLOCK_TYPE_OBJ);\n \tif (err < 0)\n \t\tgoto done;\n \n@@ -692,7 +692,7 @@ static int reftable_table_refs_for_unindexed(struct reftable_table *t,\n \t}\n \n \ttable_iter_init(ti, t);\n-\terr = table_iter_seek_start(ti, BLOCK_TYPE_REF, 0);\n+\terr = table_iter_seek_start(ti, REFTABLE_BLOCK_TYPE_REF, 0);\n \tif (err < 0)\n \t\tgoto out;\n \n@@ -748,15 +748,15 @@ int reftable_table_print_blocks(const char *tablename)\n \t} sections[] = {\n \t\t{\n \t\t\t.name = \"ref\",\n-\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t\t},\n \t\t{\n \t\t\t.name = \"obj\",\n-\t\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n \t\t},\n \t\t{\n \t\t\t.name = \"log\",\n-\t\t\t.type = BLOCK_TYPE_LOG,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t\t},\n \t};\n \tstruct reftable_block_source src = { 0 };\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex f0accfd0c32..0954c29bcf2 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -172,7 +172,7 @@ int reftable_writer_new(struct reftable_writer **out,\n \twp->write_arg = writer_arg;\n \twp->opts = opts;\n \twp->flush = flush_func;\n-\twriter_reinit_block_writer(wp, BLOCK_TYPE_REF);\n+\twriter_reinit_block_writer(wp, REFTABLE_BLOCK_TYPE_REF);\n \n \t*out = wp;\n \n@@ -347,7 +347,7 @@ int reftable_writer_add_ref(struct reftable_writer *w,\n \t\t\t    struct reftable_ref_record *ref)\n {\n \tstruct reftable_record rec = {\n-\t\t.type = BLOCK_TYPE_REF,\n+\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t\t.u = {\n \t\t\t.ref = *ref\n \t\t},\n@@ -411,13 +411,13 @@ static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n \t\t\t\t\t    struct reftable_log_record *log)\n {\n \tstruct reftable_record rec = {\n-\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t\t.u = {\n \t\t\t.log = *log,\n \t\t},\n \t};\n \tif (w->block_writer &&\n-\t    block_writer_type(w->block_writer) == BLOCK_TYPE_REF) {\n+\t    block_writer_type(w->block_writer) == REFTABLE_BLOCK_TYPE_REF) {\n \t\tint err = writer_finish_public_section(w);\n \t\tif (err < 0)\n \t\t\treturn err;\n@@ -537,7 +537,7 @@ static int writer_finish_section(struct reftable_writer *w)\n \n \t\tmax_level++;\n \t\tindex_start = w->next;\n-\t\terr = writer_reinit_block_writer(w, BLOCK_TYPE_INDEX);\n+\t\terr = writer_reinit_block_writer(w, REFTABLE_BLOCK_TYPE_INDEX);\n \t\tif (err < 0)\n \t\t\treturn err;\n \n@@ -549,7 +549,7 @@ static int writer_finish_section(struct reftable_writer *w)\n \t\tw->index_cap = 0;\n \t\tfor (i = 0; i < idx_len; i++) {\n \t\t\tstruct reftable_record rec = {\n-\t\t\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t\t\t.type = REFTABLE_BLOCK_TYPE_INDEX,\n \t\t\t\t.u = {\n \t\t\t\t\t.idx = idx[i],\n \t\t\t\t},\n@@ -614,7 +614,7 @@ static void write_object_record(void *void_arg, void *key)\n \tstruct write_record_arg *arg = void_arg;\n \tstruct obj_index_tree_node *entry = key;\n \tstruct reftable_record\n-\t\trec = { .type = BLOCK_TYPE_OBJ,\n+\t\trec = { .type = REFTABLE_BLOCK_TYPE_OBJ,\n \t\t\t.u.obj = {\n \t\t\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n \t\t\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n@@ -632,7 +632,7 @@ static void write_object_record(void *void_arg, void *key)\n \tif (arg->err < 0)\n \t\tgoto done;\n \n-\targ->err = writer_reinit_block_writer(arg->w, BLOCK_TYPE_OBJ);\n+\targ->err = writer_reinit_block_writer(arg->w, REFTABLE_BLOCK_TYPE_OBJ);\n \tif (arg->err < 0)\n \t\tgoto done;\n \n@@ -670,7 +670,7 @@ static int writer_dump_object_index(struct reftable_writer *w)\n \t\tinfix_walk(w->obj_index_tree, &update_common, &common);\n \tw->stats.object_id_len = common.max + 1;\n \n-\terr = writer_reinit_block_writer(w, BLOCK_TYPE_OBJ);\n+\terr = writer_reinit_block_writer(w, REFTABLE_BLOCK_TYPE_OBJ);\n \tif (err < 0)\n \t\treturn err;\n \n@@ -694,7 +694,7 @@ static int writer_finish_public_section(struct reftable_writer *w)\n \terr = writer_finish_section(w);\n \tif (err < 0)\n \t\treturn err;\n-\tif (typ == BLOCK_TYPE_REF && !w->opts.skip_index_objects &&\n+\tif (typ == REFTABLE_BLOCK_TYPE_REF && !w->opts.skip_index_objects &&\n \t    w->stats.ref_stats.index_blocks > 0) {\n \t\terr = writer_dump_object_index(w);\n \t\tif (err < 0)\n@@ -799,7 +799,7 @@ static int writer_flush_nonempty_block(struct reftable_writer *w)\n \t * By default, all records except for log records are padded to the\n \t * block size.\n \t */\n-\tif (!w->opts.unpadded && typ != BLOCK_TYPE_LOG)\n+\tif (!w->opts.unpadded && typ != REFTABLE_BLOCK_TYPE_LOG)\n \t\tpadding = w->opts.block_size - raw_bytes;\n \n \tbstats = writer_reftable_block_stats(w, typ);\ndiff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c\nindex e092d0bb8f8..72164df7cbc 100644\n--- a/t/unit-tests/t-reftable-block.c\n+++ b/t/unit-tests/t-reftable-block.c\n@@ -24,7 +24,7 @@ static void t_ref_block_read_write(void)\n \t\t.last_key = REFTABLE_BUF_INIT,\n \t};\n \tstruct reftable_record rec = {\n-\t\t.type = BLOCK_TYPE_REF,\n+\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t};\n \tsize_t i = 0;\n \tint ret;\n@@ -37,7 +37,7 @@ static void t_ref_block_read_write(void)\n \tcheck(block_data.buf != NULL);\n \tblock_data.len = block_size;\n \n-\tret = block_writer_init(&bw, BLOCK_TYPE_REF, (uint8_t *) block_data.buf, block_size,\n+\tret = block_writer_init(&bw, REFTABLE_BLOCK_TYPE_REF, (uint8_t *) block_data.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -119,7 +119,7 @@ static void t_log_block_read_write(void)\n \t\t.last_key = REFTABLE_BUF_INIT,\n \t};\n \tstruct reftable_record rec = {\n-\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t};\n \tsize_t i = 0;\n \tint ret;\n@@ -132,7 +132,7 @@ static void t_log_block_read_write(void)\n \tcheck(block_data.buf != NULL);\n \tblock_data.len = block_size;\n \n-\tret = block_writer_init(&bw, BLOCK_TYPE_LOG, (uint8_t *) block_data.buf, block_size,\n+\tret = block_writer_init(&bw, REFTABLE_BLOCK_TYPE_LOG, (uint8_t *) block_data.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -210,7 +210,7 @@ static void t_obj_block_read_write(void)\n \t\t.last_key = REFTABLE_BUF_INIT,\n \t};\n \tstruct reftable_record rec = {\n-\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n \t};\n \tsize_t i = 0;\n \tint ret;\n@@ -223,7 +223,7 @@ static void t_obj_block_read_write(void)\n \tcheck(block_data.buf != NULL);\n \tblock_data.len = block_size;\n \n-\tret = block_writer_init(&bw, BLOCK_TYPE_OBJ, (uint8_t *) block_data.buf, block_size,\n+\tret = block_writer_init(&bw, REFTABLE_BLOCK_TYPE_OBJ, (uint8_t *) block_data.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -294,7 +294,7 @@ static void t_index_block_read_write(void)\n \t\t.last_key = REFTABLE_BUF_INIT,\n \t};\n \tstruct reftable_record rec = {\n-\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.type = REFTABLE_BLOCK_TYPE_INDEX,\n \t\t.u.idx.last_key = REFTABLE_BUF_INIT,\n \t};\n \tsize_t i = 0;\n@@ -308,7 +308,7 @@ static void t_index_block_read_write(void)\n \tcheck(block_data.buf != NULL);\n \tblock_data.len = block_size;\n \n-\tret = block_writer_init(&bw, BLOCK_TYPE_INDEX, (uint8_t *) block_data.buf, block_size,\n+\tret = block_writer_init(&bw, REFTABLE_BLOCK_TYPE_INDEX, (uint8_t *) block_data.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -318,7 +318,7 @@ static void t_index_block_read_write(void)\n \t\tsnprintf(buf, sizeof(buf), \"branch%02\"PRIuMAX, (uintmax_t)i);\n \n \t\treftable_buf_init(&recs[i].u.idx.last_key);\n-\t\trecs[i].type = BLOCK_TYPE_INDEX;\n+\t\trecs[i].type = REFTABLE_BLOCK_TYPE_INDEX;\n \t\tcheck(!reftable_buf_addstr(&recs[i].u.idx.last_key, buf));\n \t\trecs[i].u.idx.offset = i;\n \n@@ -393,13 +393,13 @@ static void t_block_iterator(void)\n \tREFTABLE_CALLOC_ARRAY(data.buf, data.len);\n \tcheck(data.buf != NULL);\n \n-\terr = block_writer_init(&writer, BLOCK_TYPE_REF, (uint8_t *) data.buf, data.len,\n+\terr = block_writer_init(&writer, REFTABLE_BLOCK_TYPE_REF, (uint8_t *) data.buf, data.len,\n \t\t\t\t0, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!err);\n \n \tfor (size_t i = 0; i < ARRAY_SIZE(expected_refs); i++) {\n \t\texpected_refs[i] = (struct reftable_record) {\n-\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t\t\t.u.ref = {\n \t\t\t\t.value_type = REFTABLE_REF_VAL1,\n \t\t\t\t.refname = xstrfmt(\"refs/heads/branch-%02\"PRIuMAX, (uintmax_t)i),\ndiff --git a/t/unit-tests/t-reftable-merged.c b/t/unit-tests/t-reftable-merged.c\nindex fed6beb85c0..18c3251a56a 100644\n--- a/t/unit-tests/t-reftable-merged.c\n+++ b/t/unit-tests/t-reftable-merged.c\n@@ -84,7 +84,7 @@ static void t_merged_single_record(void)\n \tstruct reftable_iterator it = { 0 };\n \tint err;\n \n-\terr = merged_table_init_iter(mt, &it, BLOCK_TYPE_REF);\n+\terr = merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_REF);\n \tcheck(!err);\n \terr = reftable_iterator_seek_ref(&it, \"a\");\n \tcheck(!err);\n@@ -164,7 +164,7 @@ static void t_merged_refs(void)\n \tsize_t cap = 0;\n \tsize_t i;\n \n-\terr = merged_table_init_iter(mt, &it, BLOCK_TYPE_REF);\n+\terr = merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_REF);\n \tcheck(!err);\n \terr = reftable_iterator_seek_ref(&it, \"a\");\n \tcheck(!err);\n@@ -244,7 +244,7 @@ static void t_merged_seek_multiple_times(void)\n \tstruct reftable_merged_table *mt;\n \n \tmt = merged_table_from_records(refs, &sources, &tables, sizes, bufs, 2);\n-\tmerged_table_init_iter(mt, &it, BLOCK_TYPE_REF);\n+\tmerged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_REF);\n \n \tfor (size_t i = 0; i < 5; i++) {\n \t\tint err = reftable_iterator_seek_ref(&it, \"c\");\n@@ -320,7 +320,7 @@ static void t_merged_seek_multiple_times_without_draining(void)\n \tint err;\n \n \tmt = merged_table_from_records(refs, &sources, &tables, sizes, bufs, 2);\n-\tmerged_table_init_iter(mt, &it, BLOCK_TYPE_REF);\n+\tmerged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_REF);\n \n \terr = reftable_iterator_seek_ref(&it, \"b\");\n \tcheck(!err);\n@@ -445,7 +445,7 @@ static void t_merged_logs(void)\n \tsize_t cap = 0;\n \tsize_t i;\n \n-\terr = merged_table_init_iter(mt, &it, BLOCK_TYPE_LOG);\n+\terr = merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_LOG);\n \tcheck(!err);\n \terr = reftable_iterator_seek_log(&it, \"a\");\n \tcheck(!err);\n@@ -469,7 +469,7 @@ static void t_merged_logs(void)\n \t\tcheck(reftable_log_record_equal(want[i], &out[i],\n \t\t\t\t\t\t REFTABLE_HASH_SIZE_SHA1));\n \n-\terr = merged_table_init_iter(mt, &it, BLOCK_TYPE_LOG);\n+\terr = merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_LOG);\n \tcheck(!err);\n \terr = reftable_iterator_seek_log_at(&it, \"a\", 2);\n \tcheck(!err);\ndiff --git a/t/unit-tests/t-reftable-pq.c b/t/unit-tests/t-reftable-pq.c\nindex c128fe8616a..fb5a4eb1877 100644\n--- a/t/unit-tests/t-reftable-pq.c\n+++ b/t/unit-tests/t-reftable-pq.c\n@@ -34,7 +34,7 @@ static void t_pq_record(void)\n \tchar *last = NULL;\n \n \tfor (i = 0; i < N; i++) {\n-\t\tcheck(!reftable_record_init(&recs[i], BLOCK_TYPE_REF));\n+\t\tcheck(!reftable_record_init(&recs[i], REFTABLE_BLOCK_TYPE_REF));\n \t\trecs[i].u.ref.refname = xstrfmt(\"%02\"PRIuMAX, (uintmax_t)i);\n \t}\n \n@@ -57,7 +57,7 @@ static void t_pq_record(void)\n \t\tmerged_iter_pqueue_check(&pq);\n \n \t\tcheck(pq_entry_equal(&top, &e));\n-\t\tcheck(reftable_record_type(e.rec) == BLOCK_TYPE_REF);\n+\t\tcheck(reftable_record_type(e.rec) == REFTABLE_BLOCK_TYPE_REF);\n \t\tif (last)\n \t\t\tcheck_int(strcmp(last, e.rec->u.ref.refname), <, 0);\n \t\tlast = e.rec->u.ref.refname;\n@@ -76,7 +76,7 @@ static void t_pq_index(void)\n \tsize_t N = ARRAY_SIZE(recs), i;\n \n \tfor (i = 0; i < N; i++) {\n-\t\tcheck(!reftable_record_init(&recs[i], BLOCK_TYPE_REF));\n+\t\tcheck(!reftable_record_init(&recs[i], REFTABLE_BLOCK_TYPE_REF));\n \t\trecs[i].u.ref.refname = (char *) \"refs/heads/master\";\n \t}\n \n@@ -100,7 +100,7 @@ static void t_pq_index(void)\n \t\tmerged_iter_pqueue_check(&pq);\n \n \t\tcheck(pq_entry_equal(&top, &e));\n-\t\tcheck(reftable_record_type(e.rec) == BLOCK_TYPE_REF);\n+\t\tcheck(reftable_record_type(e.rec) == REFTABLE_BLOCK_TYPE_REF);\n \t\tcheck_int(e.index, ==, i);\n \t\tif (last)\n \t\t\tcheck_str(last, e.rec->u.ref.refname);\n@@ -117,7 +117,7 @@ static void t_merged_iter_pqueue_top(void)\n \tsize_t N = ARRAY_SIZE(recs), i;\n \n \tfor (i = 0; i < N; i++) {\n-\t\tcheck(!reftable_record_init(&recs[i], BLOCK_TYPE_REF));\n+\t\tcheck(!reftable_record_init(&recs[i], REFTABLE_BLOCK_TYPE_REF));\n \t\trecs[i].u.ref.refname = (char *) \"refs/heads/master\";\n \t}\n \ndiff --git a/t/unit-tests/t-reftable-record.c b/t/unit-tests/t-reftable-record.c\nindex 59549663736..553a0076647 100644\n--- a/t/unit-tests/t-reftable-record.c\n+++ b/t/unit-tests/t-reftable-record.c\n@@ -84,17 +84,17 @@ static void t_reftable_ref_record_comparison(void)\n {\n \tstruct reftable_record in[3] = {\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t\t\t.u.ref.refname = (char *) \"refs/heads/master\",\n \t\t\t.u.ref.value_type = REFTABLE_REF_VAL1,\n \t\t},\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t\t\t.u.ref.refname = (char *) \"refs/heads/master\",\n \t\t\t.u.ref.value_type = REFTABLE_REF_DELETION,\n \t\t},\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t\t\t.u.ref.refname = (char *) \"HEAD\",\n \t\t\t.u.ref.value_type = REFTABLE_REF_SYMREF,\n \t\t\t.u.ref.value.symref = (char *) \"refs/heads/master\",\n@@ -141,10 +141,10 @@ static void t_reftable_ref_record_roundtrip(void)\n \n \tfor (int i = REFTABLE_REF_DELETION; i < REFTABLE_NR_REF_VALUETYPES; i++) {\n \t\tstruct reftable_record in = {\n-\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t\t\t.u.ref.value_type = i,\n \t\t};\n-\t\tstruct reftable_record out = { .type = BLOCK_TYPE_REF };\n+\t\tstruct reftable_record out = { .type = REFTABLE_BLOCK_TYPE_REF };\n \t\tstruct reftable_buf key = REFTABLE_BUF_INIT;\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n@@ -198,17 +198,17 @@ static void t_reftable_log_record_comparison(void)\n {\n \tstruct reftable_record in[3] = {\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_LOG,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t\t\t.u.log.refname = (char *) \"refs/heads/master\",\n \t\t\t.u.log.update_index = 42,\n \t\t},\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_LOG,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t\t\t.u.log.refname = (char *) \"refs/heads/master\",\n \t\t\t.u.log.update_index = 22,\n \t\t},\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_LOG,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t\t\t.u.log.refname = (char *) \"refs/heads/main\",\n \t\t\t.u.log.update_index = 22,\n \t\t},\n@@ -297,7 +297,7 @@ static void t_reftable_log_record_roundtrip(void)\n \tcheck(!reftable_log_record_is_deletion(&in[2]));\n \n \tfor (size_t i = 0; i < ARRAY_SIZE(in); i++) {\n-\t\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG };\n+\t\tstruct reftable_record rec = { .type = REFTABLE_BLOCK_TYPE_LOG };\n \t\tstruct reftable_buf key = REFTABLE_BUF_INIT;\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n@@ -306,7 +306,7 @@ static void t_reftable_log_record_roundtrip(void)\n \t\t};\n \t\t/* populate out, to check for leaks. */\n \t\tstruct reftable_record out = {\n-\t\t\t.type = BLOCK_TYPE_LOG,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t\t\t.u.log = {\n \t\t\t\t.refname = xstrdup(\"old name\"),\n \t\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n@@ -384,21 +384,21 @@ static void t_reftable_obj_record_comparison(void)\n \tuint64_t offsets[] = { 0, 16, 32, 48, 64, 80, 96, 112};\n \tstruct reftable_record in[3] = {\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n \t\t\t.u.obj.hash_prefix = id_bytes,\n \t\t\t.u.obj.hash_prefix_len = 7,\n \t\t\t.u.obj.offsets = offsets,\n \t\t\t.u.obj.offset_len = 8,\n \t\t},\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n \t\t\t.u.obj.hash_prefix = id_bytes,\n \t\t\t.u.obj.hash_prefix_len = 7,\n \t\t\t.u.obj.offsets = offsets,\n \t\t\t.u.obj.offset_len = 5,\n \t\t},\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n \t\t\t.u.obj.hash_prefix = id_bytes,\n \t\t\t.u.obj.hash_prefix_len = 5,\n \t\t},\n@@ -450,13 +450,13 @@ static void t_reftable_obj_record_roundtrip(void)\n \t\t\t.len = sizeof(buffer),\n \t\t};\n \t\tstruct reftable_record in = {\n-\t\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n \t\t\t.u = {\n \t\t\t\t.obj = recs[i],\n \t\t\t},\n \t\t};\n \t\tstruct reftable_buf key = REFTABLE_BUF_INIT;\n-\t\tstruct reftable_record out = { .type = BLOCK_TYPE_OBJ };\n+\t\tstruct reftable_record out = { .type = REFTABLE_BLOCK_TYPE_OBJ };\n \t\tint n, m;\n \t\tuint8_t extra;\n \n@@ -482,17 +482,17 @@ static void t_reftable_index_record_comparison(void)\n {\n \tstruct reftable_record in[3] = {\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_INDEX,\n \t\t\t.u.idx.offset = 22,\n \t\t\t.u.idx.last_key = REFTABLE_BUF_INIT,\n \t\t},\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_INDEX,\n \t\t\t.u.idx.offset = 32,\n \t\t\t.u.idx.last_key = REFTABLE_BUF_INIT,\n \t\t},\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_INDEX,\n \t\t\t.u.idx.offset = 32,\n \t\t\t.u.idx.last_key = REFTABLE_BUF_INIT,\n \t\t},\n@@ -523,7 +523,7 @@ static void t_reftable_index_record_comparison(void)\n static void t_reftable_index_record_roundtrip(void)\n {\n \tstruct reftable_record in = {\n-\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.type = REFTABLE_BLOCK_TYPE_INDEX,\n \t\t.u.idx = {\n \t\t\t.offset = 42,\n \t\t\t.last_key = REFTABLE_BUF_INIT,\n@@ -537,7 +537,7 @@ static void t_reftable_index_record_roundtrip(void)\n \tstruct reftable_buf scratch = REFTABLE_BUF_INIT;\n \tstruct reftable_buf key = REFTABLE_BUF_INIT;\n \tstruct reftable_record out = {\n-\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.type = REFTABLE_BLOCK_TYPE_INDEX,\n \t\t.u.idx = { .last_key = REFTABLE_BUF_INIT },\n \t};\n \tint n, m;\ndiff --git a/t/unit-tests/t-reftable-table.c b/t/unit-tests/t-reftable-table.c\nindex 58b13ad496f..8dcd1da90e2 100644\n--- a/t/unit-tests/t-reftable-table.c\n+++ b/t/unit-tests/t-reftable-table.c\n@@ -106,33 +106,33 @@ static int t_table_block_iterator(void)\n \t\tuint16_t record_count;\n \t} expected_blocks[] = {\n \t\t{\n-\t\t\t.block_type = BLOCK_TYPE_REF,\n+\t\t\t.block_type = REFTABLE_BLOCK_TYPE_REF,\n \t\t\t.header_off = 24,\n \t\t\t.restart_count = 10,\n \t\t\t.record_count = 158,\n \t\t},\n \t\t{\n-\t\t\t.block_type = BLOCK_TYPE_REF,\n+\t\t\t.block_type = REFTABLE_BLOCK_TYPE_REF,\n \t\t\t.restart_count = 10,\n \t\t\t.record_count = 159,\n \t\t},\n \t\t{\n-\t\t\t.block_type = BLOCK_TYPE_REF,\n+\t\t\t.block_type = REFTABLE_BLOCK_TYPE_REF,\n \t\t\t.restart_count = 10,\n \t\t\t.record_count = 159,\n \t\t},\n \t\t{\n-\t\t\t.block_type = BLOCK_TYPE_REF,\n+\t\t\t.block_type = REFTABLE_BLOCK_TYPE_REF,\n \t\t\t.restart_count = 2,\n \t\t\t.record_count = 24,\n \t\t},\n \t\t{\n-\t\t\t.block_type = BLOCK_TYPE_INDEX,\n+\t\t\t.block_type = REFTABLE_BLOCK_TYPE_INDEX,\n \t\t\t.restart_count = 1,\n \t\t\t.record_count = 4,\n \t\t},\n \t\t{\n-\t\t\t.block_type = BLOCK_TYPE_OBJ,\n+\t\t\t.block_type = REFTABLE_BLOCK_TYPE_OBJ,\n \t\t\t.restart_count = 1,\n \t\t\t.record_count = 1,\n \t\t},\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515353","messageId":"20250331-pks-reftable-polishing-v1-16-ebed5247434c@pks.im","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im","subject":"[PATCH 16/16] reftable/table: move printing logic into test helper","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-31T08:41:34Z","receivedAt":"2025-03-31T08:41:37Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The logic to print individual blocks in a table is hosted in the\nreftable library. This is only the case due to historical reasons though\nbecause users of the library had no interfaces to read blocks one by\none. Otherwise, printing individual blocks has no place in the reftable\nlibrary given that the format will not be generic in the first place.\n\nWe have now grown a public interface to iterate through blocks contained\nin a table, and thus we can finally move the logic to print them into\nthe test helper.\n\nMove over the logic and refactor it accordingly. Note that the iterator\nalso trivially allows us to access index sections, which we previously\ndidn't print at all. This omission wasn't intentional though, so start\ndumping those sections as well so that we can assert that indices are\nwritten as expected.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/reftable-table.h         |  3 --\n reftable/table.c                  | 65 ------------------------------------\n t/helper/test-reftable.c          | 69 ++++++++++++++++++++++++++++++++++++++-\n t/t0613-reftable-write-options.sh |  9 +++++\n 4 files changed, 77 insertions(+), 69 deletions(-)\n\ndiff --git a/reftable/reftable-table.h b/reftable/reftable-table.h\nindex f0f1784c664..293fffbddc6 100644\n--- a/reftable/reftable-table.h\n+++ b/reftable/reftable-table.h\n@@ -97,9 +97,6 @@ uint64_t reftable_table_max_update_index(struct reftable_table *t);\n /* return the min_update_index for a table */\n uint64_t reftable_table_min_update_index(struct reftable_table *t);\n \n-/* print blocks onto stdout for debugging. */\n-int reftable_table_print_blocks(const char *tablename);\n-\n /*\n  * An iterator that iterates through the blocks contained in a given table.\n  */\ndiff --git a/reftable/table.c b/reftable/table.c\nindex 48f0cdfd42b..8a7581b9800 100644\n--- a/reftable/table.c\n+++ b/reftable/table.c\n@@ -740,71 +740,6 @@ uint64_t reftable_table_min_update_index(struct reftable_table *t)\n \treturn t->min_update_index;\n }\n \n-int reftable_table_print_blocks(const char *tablename)\n-{\n-\tstruct {\n-\t\tconst char *name;\n-\t\tint type;\n-\t} sections[] = {\n-\t\t{\n-\t\t\t.name = \"ref\",\n-\t\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n-\t\t},\n-\t\t{\n-\t\t\t.name = \"obj\",\n-\t\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n-\t\t},\n-\t\t{\n-\t\t\t.name = \"log\",\n-\t\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n-\t\t},\n-\t};\n-\tstruct reftable_block_source src = { 0 };\n-\tstruct reftable_table *table = NULL;\n-\tstruct table_iter ti = { 0 };\n-\tsize_t i;\n-\tint err;\n-\n-\terr = reftable_block_source_from_file(&src, tablename);\n-\tif (err < 0)\n-\t\tgoto done;\n-\n-\terr = reftable_table_new(&table, &src, tablename);\n-\tif (err < 0)\n-\t\tgoto done;\n-\n-\ttable_iter_init(&ti, table);\n-\n-\tprintf(\"header:\\n\");\n-\tprintf(\"  block_size: %d\\n\", table->block_size);\n-\n-\tfor (i = 0; i < sizeof(sections) / sizeof(*sections); i++) {\n-\t\terr = table_iter_seek_start(&ti, sections[i].type, 0);\n-\t\tif (err < 0)\n-\t\t\tgoto done;\n-\t\tif (err > 0)\n-\t\t\tcontinue;\n-\n-\t\tprintf(\"%s:\\n\", sections[i].name);\n-\n-\t\twhile (1) {\n-\t\t\tprintf(\"  - length: %u\\n\", ti.block.restart_off);\n-\t\t\tprintf(\"    restarts: %u\\n\", ti.block.restart_count);\n-\n-\t\t\terr = table_iter_next_block(&ti);\n-\t\t\tif (err < 0)\n-\t\t\t\tgoto done;\n-\t\t\tif (err > 0)\n-\t\t\t\tbreak;\n-\t\t}\n-\t}\n-\n-done:\n-\treftable_table_decref(table);\n-\ttable_iter_close(&ti);\n-\treturn err;\n-}\n-\n int reftable_table_init_table_iterator(struct reftable_table *t,\n \t\t\t\t       struct reftable_table_iterator *it)\n {\ndiff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c\nindex f8f1956f4f3..c465137826c 100644\n--- a/t/helper/test-reftable.c\n+++ b/t/helper/test-reftable.c\n@@ -2,6 +2,7 @@\n #include \"hash.h\"\n #include \"hex.h\"\n #include \"reftable/system.h\"\n+#include \"reftable/reftable-constants.h\"\n #include \"reftable/reftable-error.h\"\n #include \"reftable/reftable-merged.h\"\n #include \"reftable/reftable-stack.h\"\n@@ -20,6 +21,72 @@ static void print_help(void)\n \t       \"\\n\");\n }\n \n+static int dump_blocks(const char *tablename)\n+{\n+\tstruct reftable_table_iterator ti = { 0 };\n+\tstruct reftable_block_source src = { 0 };\n+\tstruct reftable_table *table = NULL;\n+\tuint8_t section_type = 0;\n+\tint err;\n+\n+\terr = reftable_block_source_from_file(&src, tablename);\n+\tif (err < 0)\n+\t\tgoto done;\n+\n+\terr = reftable_table_new(&table, &src, tablename);\n+\tif (err < 0)\n+\t\tgoto done;\n+\n+\terr = reftable_table_init_table_iterator(table, &ti);\n+\tif (err < 0)\n+\t\tgoto done;\n+\n+\tprintf(\"header:\\n\");\n+\tprintf(\"  block_size: %d\\n\", table->block_size);\n+\n+\twhile (1) {\n+\t\tconst struct reftable_block *block;\n+\n+\t\terr = reftable_table_iterator_next(&ti, &block);\n+\t\tif (err < 0)\n+\t\t\tgoto done;\n+\t\tif (err > 0)\n+\t\t\tbreak;\n+\n+\t\tif (block->block_type != section_type) {\n+\t\t\tconst char *section;\n+\t\t\tswitch (block->block_type) {\n+\t\t\tcase REFTABLE_BLOCK_TYPE_LOG:\n+\t\t\t\tsection = \"log\";\n+\t\t\t\tbreak;\n+\t\t\tcase REFTABLE_BLOCK_TYPE_REF:\n+\t\t\t\tsection = \"ref\";\n+\t\t\t\tbreak;\n+\t\t\tcase REFTABLE_BLOCK_TYPE_OBJ:\n+\t\t\t\tsection = \"obj\";\n+\t\t\t\tbreak;\n+\t\t\tcase REFTABLE_BLOCK_TYPE_INDEX:\n+\t\t\t\tsection = \"idx\";\n+\t\t\t\tbreak;\n+\t\t\tdefault:\n+\t\t\t\terr = -1;\n+\t\t\t\tgoto done;\n+\t\t\t}\n+\n+\t\t\tsection_type = block->block_type;\n+\t\t\tprintf(\"%s:\\n\", section);\n+\t\t}\n+\n+\t\tprintf(\"  - length: %u\\n\", block->restart_off);\n+\t\tprintf(\"    restarts: %u\\n\", block->restart_count);\n+\t}\n+\n+done:\n+\treftable_table_iterator_release(&ti);\n+\treftable_table_decref(table);\n+\treturn err;\n+}\n+\n static int dump_table(struct reftable_merged_table *mt)\n {\n \tstruct reftable_iterator it = { NULL };\n@@ -184,7 +251,7 @@ int cmd__dump_reftable(int argc, const char **argv)\n \targ = argv[1];\n \n \tif (opt_dump_blocks) {\n-\t\terr = reftable_table_print_blocks(arg);\n+\t\terr = dump_blocks(arg);\n \t} else if (opt_dump_table) {\n \t\terr = dump_reftable(arg);\n \t} else if (opt_dump_stack) {\ndiff --git a/t/t0613-reftable-write-options.sh b/t/t0613-reftable-write-options.sh\nindex e2708e11d5b..e4c7461ce9e 100755\n--- a/t/t0613-reftable-write-options.sh\n+++ b/t/t0613-reftable-write-options.sh\n@@ -93,6 +93,9 @@ test_expect_success 'many refs results in multiple blocks' '\n \t\t    restarts: 3\n \t\t  - length: 3289\n \t\t    restarts: 3\n+\t\tidx:\n+\t\t  - length: 103\n+\t\t    restarts: 1\n \t\tEOF\n \t\ttest-tool dump-reftable -b .git/reftable/*.ref >actual &&\n \t\ttest_cmp expect actual\n@@ -241,6 +244,9 @@ test_expect_success 'object index gets written by default with ref index' '\n \t\t    restarts: 1\n \t\t  - length: 80\n \t\t    restarts: 1\n+\t\tidx:\n+\t\t  - length: 55\n+\t\t    restarts: 2\n \t\tobj:\n \t\t  - length: 11\n \t\t    restarts: 1\n@@ -277,6 +283,9 @@ test_expect_success 'object index can be disabled' '\n \t\t    restarts: 1\n \t\t  - length: 80\n \t\t    restarts: 1\n+\t\tidx:\n+\t\t  - length: 55\n+\t\t    restarts: 2\n \t\tEOF\n \t\ttest-tool dump-reftable -b .git/reftable/*.ref >actual &&\n \t\ttest_cmp expect actual\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515471","messageId":"xmqq8qoja62n.fsf@gitster.g","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-14-ebed5247434c@pks.im","subject":"Re: [PATCH 14/16] reftable/table: introduce iterator for table blocks","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-04-01T22:08:00Z","receivedAt":"2025-04-01T22:08:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> +\tfor (size_t i = 0; i < nrecords; i++) {\n> +\t\trecords[i].value_type = REFTABLE_REF_VAL1,\n\n-Wcomma,error\n\n> +\t\trecords[i].refname = xstrfmt(\"refs/heads/branch-%03\"PRIuMAX, (uintmax_t) i);\n> +\t}\n\nAn overlong line.\n"},{"id":"515485","messageId":"Z-zlfVIRRrMmHU-Y@pks.im","threadId":"63222","inReplyTo":"xmqq8qoja62n.fsf@gitster.g","subject":"Re: [PATCH 14/16] reftable/table: introduce iterator for table blocks","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-02T07:21:33Z","receivedAt":"2025-04-02T07:21:37Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Apr 01, 2025 at 03:08:00PM -0700, Junio C Hamano wrote:\n> Patrick Steinhardt <ps@pks.im> writes:\n> \n> > +\tfor (size_t i = 0; i < nrecords; i++) {\n> > +\t\trecords[i].value_type = REFTABLE_REF_VAL1,\n> \n> -Wcomma,error\n\nHah, good that we've now got the warning.\n\n> > +\t\trecords[i].refname = xstrfmt(\"refs/heads/branch-%03\"PRIuMAX, (uintmax_t) i);\n> > +\t}\n> \n> An overlong line.\n\nThanks for these two, fixed both of them locally. Will wait a bit for\nmore reviews though before sending out a new version.\n\nPatrick\n"},{"id":"515523","messageId":"i2zpjkbunx5ilmdtjns6dhsuuc7vdncpz3se3my43sajqqjsrf@c7jj2h3z7f3o","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-3-ebed5247434c@pks.im","subject":"Re: [PATCH 03/16] reftable/blocksource: consolidate code into a single file","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-04-02T17:42:51Z","receivedAt":"2025-04-02T17:46:40Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/31 10:41AM, Patrick Steinhardt wrote:\n> The code that implements block sources is distributed across a couple of\n> files even though. Consolidate all of it into \"reftable/blocksource.c\"\n\neven though... ? I think you meant to followup with something here.\n\n> and its accompanying header so that it is easier to locate and more self\n> contained.\n\nIt looks like some of the block source function get renamed in the\nprocess. It might we worth mentioning that in the commit message too.\nOther than that, everything else in this patch seems to be a simple\nreorganization. Looks good!\n\n-Justin\n"},{"id":"515524","messageId":"pole54ty25p4w2whskjpdpld4okwwbiiqufzzjwqzd3n7bqds5@foo67s5pgk7r","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-4-ebed5247434c@pks.im","subject":"Re: [PATCH 04/16] reftable/block: simplify how we track restart points","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-04-02T18:08:15Z","receivedAt":"2025-04-02T18:12:05Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/31 10:41AM, Patrick Steinhardt wrote:\n> Restart points record the location of reftable records that do not use\n> prefix compression and are used to perform a binary search inside of a\n> block. These restart points are encoded at the end of a block, between\n> the record data and the footer of a table.\n> \n> The block structure contains three different variables related to these\n> restart points:\n> \n>   - The block length contains the length of the reftable block up to the\n>     restart points.\n> \n>   - The restart count contains the number of restart points contained in\n>     the block.\n> \n>   - The restart bytes variable tracks where the restart point data\n>     begins.\n> \n> Tracking all three of these variables is unnecessary though as the data\n> can be derived from one another: the block length without restart points\n> is the exact same as the offset of the restart count data, which we\n> already track via the `restart_bytes` data.\n> \n> Refactor the code so that we track the location of restart bytes not as\n> a pointer, but instead as an offset. This allows us to trivially get rid\n> of the `block_len` variable as described above. This avoids having the\n> confusing `block_len` variable and allows us to do less bookkeeping\n> overall.\n> \n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  reftable/block.c | 25 ++++++++++++-------------\n>  reftable/block.h |  8 +++++---\n>  reftable/table.c |  2 +-\n>  3 files changed, 18 insertions(+), 17 deletions(-)\n> \n> diff --git a/reftable/block.c b/reftable/block.c\n> index 97740187259..f2567a8f0fd 100644\n> --- a/reftable/block.c\n> +++ b/reftable/block.c\n> @@ -216,10 +216,9 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n>  \tuint32_t full_block_size = table_block_size;\n>  \tuint8_t typ = block->data[header_off];\n>  \tuint32_t sz = reftable_get_be24(block->data + header_off + 1);\n> -\tint err = 0;\n> -\tuint16_t restart_count = 0;\n> -\tuint32_t restart_start = 0;\n> -\tuint8_t *restart_bytes = NULL;\n> +\tuint16_t restart_count;\n> +\tuint32_t restart_off;\n> +\tint err;\n>  \n>  \tblock_source_return_block(&br->block);\n>  \n> @@ -300,8 +299,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n>  \t}\n>  \n>  \trestart_count = reftable_get_be16(block->data + sz - 2);\n> -\trestart_start = sz - 2 - 3 * restart_count;\n> -\trestart_bytes = block->data + restart_start;\n> +\trestart_off = sz - 2 - 3 * restart_count;\n\nOk, since `restart_bytes` is derived from `restart_start`, instead of\nhaving both, we just track the offset which is `restart_start` and rename\nit to `restart_off`. Make sense.\n\n>  \t/* transfer ownership. */\n>  \tbr->block = *block;\n> @@ -309,11 +307,12 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n>  \tblock->len = 0;\n>  \n>  \tbr->hash_size = hash_size;\n> -\tbr->block_len = restart_start;\n> +\tbr->restart_off = restart_off;\n\nThe `block_len` is always the size of the restart offset. I assume we\nrename it here to keep naming more consistent.\n\nThe remainder of this patch looks good.\n\n>  \tbr->full_block_size = full_block_size;\n>  \tbr->header_off = header_off;\n>  \tbr->restart_count = restart_count;\n> -\tbr->restart_bytes = restart_bytes;\n> +\n> +\terr = 0;\n>  \n>  done:\n>  \treturn err;\n> @@ -337,7 +336,7 @@ int block_reader_first_key(const struct block_reader *br, struct reftable_buf *k\n>  \tint off = br->header_off + 4, n;\n>  \tstruct string_view in = {\n>  \t\t.buf = br->block.data + off,\n> -\t\t.len = br->block_len - off,\n> +\t\t.len = br->restart_off - off,\n>  \t};\n>  \tuint8_t extra = 0;\n>  \n> @@ -354,13 +353,13 @@ int block_reader_first_key(const struct block_reader *br, struct reftable_buf *k\n>  \n>  static uint32_t block_reader_restart_offset(const struct block_reader *br, size_t idx)\n>  {\n> -\treturn reftable_get_be24(br->restart_bytes + 3 * idx);\n> +\treturn reftable_get_be24(br->block.data + br->restart_off + 3 * idx);\n>  }\n>  \n>  void block_iter_seek_start(struct block_iter *it, const struct block_reader *br)\n>  {\n>  \tit->block = br->block.data;\n> -\tit->block_len = br->block_len;\n> +\tit->block_len = br->restart_off;\n>  \tit->hash_size = br->hash_size;\n>  \treftable_buf_reset(&it->last_key);\n>  \tit->next_off = br->header_off + 4;\n> @@ -378,7 +377,7 @@ static int restart_needle_less(size_t idx, void *_args)\n>  \tuint32_t off = block_reader_restart_offset(args->reader, idx);\n>  \tstruct string_view in = {\n>  \t\t.buf = args->reader->block.data + off,\n> -\t\t.len = args->reader->block_len - off,\n> +\t\t.len = args->reader->restart_off - off,\n>  \t};\n>  \tuint64_t prefix_len, suffix_len;\n>  \tuint8_t extra;\n> @@ -505,7 +504,7 @@ int block_iter_seek_key(struct block_iter *it, const struct block_reader *br,\n>  \telse\n>  \t\tit->next_off = br->header_off + 4;\n>  \tit->block = br->block.data;\n> -\tit->block_len = br->block_len;\n> +\tit->block_len = br->restart_off;\n>  \tit->hash_size = br->hash_size;\n>  \n>  \terr = reftable_record_init(&rec, block_reader_type(br));\n> diff --git a/reftable/block.h b/reftable/block.h\n> index 203b07d9a44..b78f322e646 100644\n> --- a/reftable/block.h\n> +++ b/reftable/block.h\n> @@ -79,10 +79,12 @@ struct block_reader {\n>  \tunsigned char *uncompressed_data;\n>  \tsize_t uncompressed_cap;\n>  \n> -\t/* size of the data, excluding restart data. */\n> -\tuint32_t block_len;\n> -\tuint8_t *restart_bytes;\n> +\t/*\n> +\t * Restart point data. Restart points are located after the block's\n> +\t * record data.\n> +\t */\n>  \tuint16_t restart_count;\n> +\tuint32_t restart_off;\n>  \n>  \t/* size of the data in the file. For log blocks, this is the compressed\n>  \t * size. */\n> diff --git a/reftable/table.c b/reftable/table.c\n> index d18e17b0d44..ec84545707c 100644\n> --- a/reftable/table.c\n> +++ b/reftable/table.c\n> @@ -838,7 +838,7 @@ int reftable_table_print_blocks(const char *tablename)\n>  \t\tprintf(\"%s:\\n\", sections[i].name);\n>  \n>  \t\twhile (1) {\n> -\t\t\tprintf(\"  - length: %u\\n\", ti.br.block_len);\n> +\t\t\tprintf(\"  - length: %u\\n\", ti.br.restart_off);\n>  \t\t\tprintf(\"    restarts: %u\\n\", ti.br.restart_count);\n>  \n>  \t\t\terr = table_iter_next_block(&ti);\n> \n> -- \n> 2.49.0.604.gff1f9ca942.dirty\n> \n> \n"},{"id":"515536","messageId":"umaolgemnks4g4cgwgnurl7alic2a3pzrfx2ws3s7uxypiedia@7mwqxvhw2zxg","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-5-ebed5247434c@pks.im","subject":"Re: [PATCH 05/16] reftable/table: move reading block into block reader","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-04-02T20:13:03Z","receivedAt":"2025-04-02T20:16:53Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/31 10:41AM, Patrick Steinhardt wrote:\n> The logic to read blocks from a reftable is scattered across both the\n> table and the block subsystems. Besides causing somewhat fuzzy\n> responsibilities, it also means that we have to awkwardly pass around\n> the ownership of blocks between the subsystems.\n> \n> Refactor the code so that we stop passing the block when initializing a\n> reader, but instead by passing in the block source plus the offset at\n> which we we're supposed to read a block. Like this, the ownership of the\n\ns/we we're/we're/\n\n> block itself doesn't need to get handed over as the block reader is the\n> one owning the block right from the start.\n> \n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  reftable/block.c                | 87 ++++++++++++++++++++++++++---------------\n>  reftable/block.h                |  8 ++--\n>  reftable/table.c                | 65 +++---------------------------\n>  t/unit-tests/t-reftable-block.c | 76 ++++++++++++++++++-----------------\n>  4 files changed, 107 insertions(+), 129 deletions(-)\n> \n> diff --git a/reftable/block.c b/reftable/block.c\n> index f2567a8f0fd..2517108b8ef 100644\n> --- a/reftable/block.c\n> +++ b/reftable/block.c\n> @@ -209,31 +209,57 @@ int block_writer_finish(struct block_writer *w)\n>  \treturn w->next;\n>  }\n>  \n> -int block_reader_init(struct block_reader *br, struct reftable_block *block,\n> -\t\t      uint32_t header_off, uint32_t table_block_size,\n> -\t\t      uint32_t hash_size)\n> +static int read_block(struct reftable_block_source *source,\n> +\t\t      struct reftable_block *dest, uint64_t off,\n> +\t\t      uint32_t sz)\n>  {\n> +\tsize_t size = block_source_size(source);\n> +\tblock_source_return_block(dest);\n> +\tif (off >= size)\n> +\t\treturn 0;\n> +\tif (off + sz > size)\n> +\t\tsz = size - off;\n> +\treturn block_source_read_block(source, dest, off, sz);\n> +}\n> +\n> +int block_reader_init(struct block_reader *br,\n> +\t\t      struct reftable_block_source *source,\n> +\t\t      uint32_t offset, uint32_t header_size,\n> +\t\t      uint32_t table_block_size, uint32_t hash_size)\n> +{\n> +\tuint32_t guess_block_size = table_block_size ?\n> +\t\ttable_block_size : DEFAULT_BLOCK_SIZE;\n\nOut of curiousity, in what scenarios would the table not know the block\nsize and we have to rely on the guess?\n\n>  \tuint32_t full_block_size = table_block_size;\n> -\tuint8_t typ = block->data[header_off];\n> -\tuint32_t sz = reftable_get_be24(block->data + header_off + 1);\n>  \tuint16_t restart_count;\n>  \tuint32_t restart_off;\n> +\tuint32_t block_size;\n> +\tuint8_t block_type;\n>  \tint err;\n>  \n> -\tblock_source_return_block(&br->block);\n> +\terr = read_block(source, &br->block, offset, guess_block_size);\n> +\tif (err < 0)\n> +\t\tgoto done;\n\nOk, so now `block_reader_init()` handles reading the block itself and no\nlonger relies on the read block being provided to it. This makes block\nownership more self-contained and clear.\n\n> -\tif (!reftable_is_block_type(typ)) {\n> -\t\terr =  REFTABLE_FORMAT_ERROR;\n> +\tblock_type = br->block.data[header_size];\n> +\tif (!reftable_is_block_type(block_type)) {\n> +\t\terr = REFTABLE_FORMAT_ERROR;\n>  \t\tgoto done;\n>  \t}\n>  \n> -\tif (typ == BLOCK_TYPE_LOG) {\n> -\t\tuint32_t block_header_skip = 4 + header_off;\n> -\t\tuLong dst_len = sz - block_header_skip;\n> -\t\tuLong src_len = block->len - block_header_skip;\n> +\tblock_size = reftable_get_be24(br->block.data + header_size + 1);\n> +\tif (block_size > guess_block_size) {\n> +\t\terr = read_block(source, &br->block, offset, block_size);\n> +\t\tif (err < 0)\n> +\t\t\tgoto done;\n> +\t}\n\nInstead of relying on `table_init_block_reader()` to determine if\n`guess_block_size` was correct and reread the block, this is now handled\nas part of the initial `block_reader_init()`. Make sense.\n\n[snip]\n"},{"id":"515537","messageId":"y3lbo4dnuv46kgy7fhzcksdea2pdm6nsy43tjai6tcgd6b3w5u@w5zqty5tgpmt","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-6-ebed5247434c@pks.im","subject":"Re: [PATCH 06/16] reftable/block: rename `block` to `block_data`","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-04-02T20:26:26Z","receivedAt":"2025-04-02T20:30:16Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/31 10:41AM, Patrick Steinhardt wrote:\n> The `reftable_block` structure associates a byte slice with a block\n> source. As such it only holds the data of a reftable block without\n> actually encoding any of the details for how to access that data.\n> \n> Rename the structure to instead be called `reftable_block_data`. Besides\n> clarifying that this really only holds data, it also allows us to rename\n> the `reftable_block_reader` to `reftable_block` in the next commit, as\n> this is the structure that actually encapsulates access to the reftable\n> blocks.\n> \n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  reftable/block.c                    | 10 ++++-----\n>  reftable/block.h                    |  2 +-\n>  reftable/blocksource.c              | 44 ++++++++++++++++++-------------------\n>  reftable/blocksource.h              | 10 ++++-----\n>  reftable/iter.c                     |  4 ++--\n>  reftable/reftable-blocksource.h     | 14 ++++++------\n>  reftable/table.c                    | 14 ++++++------\n>  t/unit-tests/t-reftable-readwrite.c | 10 ++++-----\n>  8 files changed, 54 insertions(+), 54 deletions(-)\n[snip]  \n> diff --git a/reftable/block.h b/reftable/block.h\n> index 6afb1b2952a..c55d47c3c52 100644\n> --- a/reftable/block.h\n> +++ b/reftable/block.h\n> @@ -71,7 +71,7 @@ struct block_reader {\n>  \tuint32_t header_off;\n>  \n>  \t/* the memory block */\n> -\tstruct reftable_block block;\n> +\tstruct reftable_block_data block;\n\nI wonder if we should rename the `reftable_block_data` here from `block`\nto `data`. In the next commit we rename `block_reader` to `block` which\nleads to `block->block` in some places which I think looks a bit funny.\n\n-Justin\n"},{"id":"515538","messageId":"y47rmb3ahghwvhdukfj27fihddmg57rqpfcqmaylfejqjjxvrr@luc7xuu5qiza","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-7-ebed5247434c@pks.im","subject":"Re: [PATCH 07/16] reftable/block: rename `block_reader` to `reftable_block`","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-04-02T20:39:57Z","receivedAt":"2025-04-02T20:43:47Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/31 10:41AM, Patrick Steinhardt wrote:\n> The `block_reader` structure is used to access parsed data of a reftable\n> block. The structure is currently treated as an internal implementation\n> detail and not exposed via our public interfaces. The functionality\n> provided by the structure is useful to external users of the reftable\n> library though, for example when implementing consistency checks that\n> need to scan through the blocks manually.\n> \n> Rename the structure to `reftable_block` now that the name has been made\n> available in the preceding commit. This name is in line with the naming\n> schema used for other data structures like `reftable_table` in that it\n> describes the underlying entity that it provides access to.\n> \n> The new data structure isn't yet exposed via the public interface, which\n> is left for a subsequent commit.\n> \n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  reftable/block.c                | 142 ++++++++++++++++++++--------------------\n>  reftable/block.h                |  29 ++++----\n>  reftable/iter.c                 |   9 ++-\n>  reftable/iter.h                 |   2 +-\n>  reftable/table.c                |  46 ++++++-------\n>  reftable/table.h                |   8 ++-\n>  t/unit-tests/t-reftable-block.c | 102 ++++++++++++++---------------\n>  7 files changed, 172 insertions(+), 166 deletions(-)\n> \n> diff --git a/reftable/block.c b/reftable/block.c\n> index ad162ecdbf6..d188665388d 100644\n> --- a/reftable/block.c\n> +++ b/reftable/block.c\n> @@ -222,10 +222,10 @@ static int read_block(struct reftable_block_source *source,\n>  \treturn block_source_read_data(source, dest, off, sz);\n>  }\n>  \n> -int block_reader_init(struct block_reader *br,\n> -\t\t      struct reftable_block_source *source,\n> -\t\t      uint32_t offset, uint32_t header_size,\n> -\t\t      uint32_t table_block_size, uint32_t hash_size)\n> +int reftable_block_init(struct reftable_block *block,\n> +\t\t\tstruct reftable_block_source *source,\n> +\t\t\tuint32_t offset, uint32_t header_size,\n> +\t\t\tuint32_t table_block_size, uint32_t hash_size)\n>  {\n>  \tuint32_t guess_block_size = table_block_size ?\n>  \t\ttable_block_size : DEFAULT_BLOCK_SIZE;\n> @@ -236,19 +236,19 @@ int block_reader_init(struct block_reader *br,\n>  \tuint8_t block_type;\n>  \tint err;\n>  \n> -\terr = read_block(source, &br->block, offset, guess_block_size);\n> +\terr = read_block(source, &block->block, offset, guess_block_size);\n>  \tif (err < 0)\n>  \t\tgoto done;\n>  \n> -\tblock_type = br->block.data[header_size];\n> +\tblock_type = block->block.data[header_size];\n\nAh, in the previous patch I suggested renaming the `block` field to\n`data`, but I guess that would lead to `block->data.data` here and would\nalso be a bit funny. It's probably fine to just leave it as-is then.\n\nThe rest of this patch is just a refactor to rename types from\n`block_reader` to `reftable_block` and associated variable names to\nmatch which all looks correct.\n\n-Justin\n"},{"id":"515539","messageId":"u4h53ukh5uxk4r4zy5iyniob4xf7qmtem7j2kp45sknfbljwuv@ntdojrh6aakb","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-10-ebed5247434c@pks.im","subject":"Re: [PATCH 10/16] reftable/block: store block pointer in the block iterator","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-04-02T20:56:30Z","receivedAt":"2025-04-02T21:00:20Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/31 10:41AM, Patrick Steinhardt wrote:\n> The block iterator requires access to a bunch of data from the\n> underlying `reftable_block` that it is iterating over. This data is\n> stored by copying over relevant data into a separate set of variables.\n> This has multiple downsides:\n> \n>   - We require more storage space than necessary. This is more of a\n>     theoretical issue as we shouldn't ever have many blocks.\n> \n>   - We have to perform more bookkeeping, and the variable names are\n>     inconsistent across the two data structures. This can lead to some\n>     confusion.\n> \n>   - The lifetime of the block iterator is tied to the block anyway, but\n>     we hide that a bit by only storing pointers into the block.\n\ns/into/in/\n\n> There isn't really any good reason why we rip out parts of the block\n> instead of storing a pointer to the block itself.\n> \n> Refactor the code to do so. Despite being simpler, it also allows us to\n> decouple the lifetime of the block iterator from seeking in a subsequent\n> commit.\n> \n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  reftable/block.c | 22 ++++++++--------------\n>  reftable/block.h |  4 +---\n>  2 files changed, 9 insertions(+), 17 deletions(-)\n> \n[snip]\n> diff --git a/reftable/block.h b/reftable/block.h\n> index 4f7f29028c4..268d5a1e005 100644\n> --- a/reftable/block.h\n> +++ b/reftable/block.h\n> @@ -67,9 +67,7 @@ void block_writer_release(struct block_writer *bw);\n>  struct block_iter {\n>  \t/* offset within the block of the next entry to read. */\n>  \tuint32_t next_off;\n> -\tconst unsigned char *block;\n> -\tsize_t block_len;\n> -\tuint32_t hash_size;\n> +\tconst struct reftable_block *block;\n\nThis is much simpler. Nice!\n\n>  \t/* key for last entry we read. */\n>  \tstruct reftable_buf last_key;\n> \n> -- \n> 2.49.0.604.gff1f9ca942.dirty\n> \n> \n"},{"id":"515540","messageId":"5dbbo5wiszgx74ijinu7xkf7ptddoubkzt53sc6cg5ak357fja@x7v2kra4ny32","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-11-ebed5247434c@pks.im","subject":"Re: [PATCH 11/16] reftable/block: make block iterators reseekable","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-04-02T21:24:53Z","receivedAt":"2025-04-02T21:28:42Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/31 10:41AM, Patrick Steinhardt wrote:\n> Refactor the block iterators so that initialization and seeking are\n> different from one another. This makes the iterator trivially reseekable\n> by storing the pointer to the block at initialization time, which we can\n> then reuse on ever seek.\n\ns/ever/every/\n\n> This refactoring prepares the code for exposing a `reftable_iterator`\n> interface for blocks in a subsequent commit. Callsites are adjusted\n> accordingly.\n> \n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  reftable/block.c                | 23 +++++++++++++----------\n>  reftable/block.h                | 21 ++++++++++++++++-----\n>  reftable/iter.c                 |  2 +-\n>  reftable/table.c                | 11 +++++++----\n>  t/unit-tests/t-reftable-block.c | 30 +++++++++++++++---------------\n>  5 files changed, 52 insertions(+), 35 deletions(-)\n> \n> diff --git a/reftable/block.c b/reftable/block.c\n> index 576c6caf59b..6e5c1191f5f 100644\n> --- a/reftable/block.c\n> +++ b/reftable/block.c\n> @@ -381,11 +381,16 @@ static uint32_t block_restart_offset(const struct reftable_block *b, size_t idx)\n>  \treturn reftable_get_be24(b->block.data + b->restart_off + 3 * idx);\n>  }\n>  \n> -void block_iter_seek_start(struct block_iter *it, const struct reftable_block *block)\n> +void block_iter_init(struct block_iter *it, const struct reftable_block *block)\n>  {\n>  \tit->block = block;\n> +\tblock_iter_seek_start(it);\n> +}\n> +\n> +void block_iter_seek_start(struct block_iter *it)\n> +{\n>  \treftable_buf_reset(&it->last_key);\n> -\tit->next_off = block->header_off + 4;\n> +\tit->next_off = it->block->header_off + 4;\n>  }\n>  \n>  struct restart_needle_less_args {\n> @@ -473,12 +478,11 @@ void block_iter_close(struct block_iter *it)\n>  \treftable_buf_release(&it->scratch);\n>  }\n>  \n> -int block_iter_seek_key(struct block_iter *it, const struct reftable_block *block,\n> -\t\t\tstruct reftable_buf *want)\n> +int block_iter_seek_key(struct block_iter *it, struct reftable_buf *want)\n\nAh so now that we store the `reftable_block` directly as part of\n`block_iter`, passing the block in `block_iter_seek_key()` becomes\nredundant. Nice!\n\n>  {\n>  \tstruct restart_needle_less_args args = {\n>  \t\t.needle = *want,\n> -\t\t.block = block,\n> +\t\t.block = it->block,\n>  \t};\n>  \tstruct reftable_record rec;\n>  \tint err = 0;\n> @@ -496,7 +500,7 @@ int block_iter_seek_key(struct block_iter *it, const struct reftable_block *bloc\n>  \t * restart point. While that works alright, we would end up scanning\n>  \t * too many record.\n>  \t */\n> -\ti = binsearch(block->restart_count, &restart_needle_less, &args);\n> +\ti = binsearch(it->block->restart_count, &restart_needle_less, &args);\n>  \tif (args.error) {\n>  \t\terr = REFTABLE_FORMAT_ERROR;\n>  \t\tgoto done;\n> @@ -521,12 +525,11 @@ int block_iter_seek_key(struct block_iter *it, const struct reftable_block *bloc\n>  \t *     starting from the preceding restart point.\n>  \t */\n>  \tif (i > 0)\n> -\t\tit->next_off = block_restart_offset(block, i - 1);\n> +\t\tit->next_off = block_restart_offset(it->block, i - 1);\n>  \telse\n> -\t\tit->next_off = block->header_off + 4;\n> -\tit->block = block;\n> +\t\tit->next_off = it->block->header_off + 4;\n>  \n> -\terr = reftable_record_init(&rec, reftable_block_type(block));\n> +\terr = reftable_record_init(&rec, reftable_block_type(it->block));\n>  \tif (err < 0)\n>  \t\tgoto done;\n>  \n> diff --git a/reftable/block.h b/reftable/block.h\n> index 268d5a1e005..1bfd44f56aa 100644\n> --- a/reftable/block.h\n> +++ b/reftable/block.h\n> @@ -79,12 +79,23 @@ struct block_iter {\n>  \t.scratch = REFTABLE_BUF_INIT, \\\n>  }\n>  \n> -/* Position `it` at start of the block */\n> -void block_iter_seek_start(struct block_iter *it, const struct reftable_block *block);\n> +/*\n> + * Initialize the block iterator with the given block. The iterator will be\n> + * positioned at the first record contained in the block. The block must remain\n> + * valid until the end of the iterator's lifetime. It is valid to re-initialize\n> + * iterators multiple times.\n> + */\n> +void block_iter_init(struct block_iter *it, const struct reftable_block *block);\n> +\n> +/* Position the initialized iterator at the first record of its block. */\n> +void block_iter_seek_start(struct block_iter *it);\n>  \n> -/* Position `it` to the `want` key in the block */\n> -int block_iter_seek_key(struct block_iter *it, const struct reftable_block *block,\n> -\t\t\tstruct reftable_buf *want);\n> +/*\n> + * Position the initialized iterator at the desired record key. It is not an\n> + * error in case the record cannot be found. If so, a subsequent call to\n> + * `block_iter_next()` will indicate that the iterator is exhausted.\n> + */\n> +int block_iter_seek_key(struct block_iter *it, struct reftable_buf *want);\n>  \n>  /* return < 0 for error, 0 for OK, > 0 for EOF. */\n>  int block_iter_next(struct block_iter *it, struct reftable_record *rec);\n> diff --git a/reftable/iter.c b/reftable/iter.c\n> index 74684de8f6a..7a7e8aa4d7e 100644\n> --- a/reftable/iter.c\n> +++ b/reftable/iter.c\n> @@ -139,7 +139,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n>  \t\t/* indexed block does not exist. */\n>  \t\treturn REFTABLE_FORMAT_ERROR;\n>  \t}\n> -\tblock_iter_seek_start(&it->cur, &it->block);\n> +\tblock_iter_init(&it->cur, &it->block);\n>  \treturn 0;\n>  }\n>  \n> diff --git a/reftable/table.c b/reftable/table.c\n> index 3f2e70caac4..5422ed6769c 100644\n> --- a/reftable/table.c\n> +++ b/reftable/table.c\n> @@ -208,7 +208,7 @@ static int table_iter_next_block(struct table_iter *ti)\n>  \n>  \tti->block_off = next_block_off;\n>  \tti->is_finished = 0;\n> -\tblock_iter_seek_start(&ti->bi, &ti->block);\n> +\tblock_iter_init(&ti->bi, &ti->block);\n>  \n>  \treturn 0;\n>  }\n> @@ -256,7 +256,7 @@ static int table_iter_seek_to(struct table_iter *ti, uint64_t off, uint8_t typ)\n>  \n>  \tti->typ = reftable_block_type(&ti->block);\n>  \tti->block_off = off;\n> -\tblock_iter_seek_start(&ti->bi, &ti->block);\n> +\tblock_iter_init(&ti->bi, &ti->block);\n>  \tti->is_finished = 0;\n>  \treturn 0;\n>  }\n> @@ -349,7 +349,8 @@ static int table_iter_seek_linear(struct table_iter *ti,\n>  \t * the wanted key inside of it. If the block does not contain our key\n>  \t * we know that the corresponding record does not exist.\n>  \t */\n> -\terr = block_iter_seek_key(&ti->bi, &ti->block, &want_key);\n> +\tblock_iter_init(&ti->bi, &ti->block);\n> +\terr = block_iter_seek_key(&ti->bi, &want_key);\n>  \tif (err < 0)\n>  \t\tgoto done;\n>  \terr = 0;\n> @@ -417,7 +418,9 @@ static int table_iter_seek_indexed(struct table_iter *ti,\n>  \t\tif (err != 0)\n>  \t\t\tgoto done;\n>  \n> -\t\terr = block_iter_seek_key(&ti->bi, &ti->block, &want_index.u.idx.last_key);\n> +\t\tblock_iter_init(&ti->bi, &ti->block);\n> +\n> +\t\terr = block_iter_seek_key(&ti->bi, &want_index.u.idx.last_key);\n>  \t\tif (err < 0)\n>  \t\t\tgoto done;\n>  \n> diff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c\n> index e36ed7ac576..c4ced39a73b 100644\n> --- a/t/unit-tests/t-reftable-block.c\n> +++ b/t/unit-tests/t-reftable-block.c\n> @@ -66,7 +66,8 @@ static void t_ref_block_read_write(void)\n>  \tblock_source_from_buf(&source ,&block_data);\n>  \treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n>  \n> -\tblock_iter_seek_start(&it, &block);\n> +\tblock_iter_init(&it, &block);\n> +\tblock_iter_seek_start(&it);\n\nIf I'm understanding correctly, `block_iter_init()` already invokes\n`block_iter_seek_start()`. Why do we need to invoke\n`block_iter_seek_start()` again?\n\n>  \n>  \tfor (i = 0; ; i++) {\n>  \t\tret = block_iter_next(&it, &rec);\n> @@ -79,10 +80,9 @@ static void t_ref_block_read_write(void)\n>  \t}\n>  \n>  \tfor (i = 0; i < N; i++) {\n> -\t\tblock_iter_reset(&it);\n>  \t\treftable_record_key(&recs[i], &want);\n>  \n> -\t\tret = block_iter_seek_key(&it, &block, &want);\n> +\t\tret = block_iter_seek_key(&it, &want);\n>  \t\tcheck_int(ret, ==, 0);\n>  \n>  \t\tret = block_iter_next(&it, &rec);\n> @@ -91,7 +91,7 @@ static void t_ref_block_read_write(void)\n>  \t\tcheck(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1));\n>  \n>  \t\twant.len--;\n> -\t\tret = block_iter_seek_key(&it, &block, &want);\n> +\t\tret = block_iter_seek_key(&it, &want);\n>  \t\tcheck_int(ret, ==, 0);\n>  \n>  \t\tret = block_iter_next(&it, &rec);\n> @@ -156,7 +156,8 @@ static void t_log_block_read_write(void)\n>  \tblock_source_from_buf(&source, &block_data);\n>  \treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n>  \n> -\tblock_iter_seek_start(&it, &block);\n> +\tblock_iter_init(&it, &block);\n> +\tblock_iter_seek_start(&it);\n>  \n>  \tfor (i = 0; ; i++) {\n>  \t\tret = block_iter_next(&it, &rec);\n> @@ -169,11 +170,10 @@ static void t_log_block_read_write(void)\n>  \t}\n>  \n>  \tfor (i = 0; i < N; i++) {\n> -\t\tblock_iter_reset(&it);\n>  \t\treftable_buf_reset(&want);\n>  \t\tcheck(!reftable_buf_addstr(&want, recs[i].u.log.refname));\n>  \n> -\t\tret = block_iter_seek_key(&it, &block, &want);\n> +\t\tret = block_iter_seek_key(&it, &want);\n>  \t\tcheck_int(ret, ==, 0);\n>  \n>  \t\tret = block_iter_next(&it, &rec);\n> @@ -182,7 +182,7 @@ static void t_log_block_read_write(void)\n>  \t\tcheck(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1));\n>  \n>  \t\twant.len--;\n> -\t\tret = block_iter_seek_key(&it, &block, &want);\n> +\t\tret = block_iter_seek_key(&it, &want);\n>  \t\tcheck_int(ret, ==, 0);\n>  \n>  \t\tret = block_iter_next(&it, &rec);\n> @@ -249,7 +249,8 @@ static void t_obj_block_read_write(void)\n>  \tblock_source_from_buf(&source, &block_data);\n>  \treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n>  \n> -\tblock_iter_seek_start(&it, &block);\n> +\tblock_iter_init(&it, &block);\n> +\tblock_iter_seek_start(&it);\n>  \n>  \tfor (i = 0; ; i++) {\n>  \t\tret = block_iter_next(&it, &rec);\n> @@ -262,10 +263,9 @@ static void t_obj_block_read_write(void)\n>  \t}\n>  \n>  \tfor (i = 0; i < N; i++) {\n> -\t\tblock_iter_reset(&it);\n>  \t\treftable_record_key(&recs[i], &want);\n>  \n> -\t\tret = block_iter_seek_key(&it, &block, &want);\n> +\t\tret = block_iter_seek_key(&it, &want);\n>  \t\tcheck_int(ret, ==, 0);\n>  \n>  \t\tret = block_iter_next(&it, &rec);\n> @@ -334,7 +334,8 @@ static void t_index_block_read_write(void)\n>  \tblock_source_from_buf(&source, &block_data);\n>  \treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n>  \n> -\tblock_iter_seek_start(&it, &block);\n> +\tblock_iter_init(&it, &block);\n> +\tblock_iter_seek_start(&it);\n>  \n>  \tfor (i = 0; ; i++) {\n>  \t\tret = block_iter_next(&it, &rec);\n> @@ -347,10 +348,9 @@ static void t_index_block_read_write(void)\n>  \t}\n>  \n>  \tfor (i = 0; i < N; i++) {\n> -\t\tblock_iter_reset(&it);\n>  \t\treftable_record_key(&recs[i], &want);\n>  \n> -\t\tret = block_iter_seek_key(&it, &block, &want);\n> +\t\tret = block_iter_seek_key(&it, &want);\n>  \t\tcheck_int(ret, ==, 0);\n>  \n>  \t\tret = block_iter_next(&it, &rec);\n> @@ -359,7 +359,7 @@ static void t_index_block_read_write(void)\n>  \t\tcheck(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1));\n>  \n>  \t\twant.len--;\n> -\t\tret = block_iter_seek_key(&it, &block, &want);\n> +\t\tret = block_iter_seek_key(&it, &want);\n>  \t\tcheck_int(ret, ==, 0);\n>  \n>  \t\tret = block_iter_next(&it, &rec);\n> \n> -- \n> 2.49.0.604.gff1f9ca942.dirty\n> \n> \n"},{"id":"515541","messageId":"q6shqgjoe5vlt3fduj6cq6473plz7mmlmnmyuwft7huuod5hhn@je7sijo3v54z","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-14-ebed5247434c@pks.im","subject":"Re: [PATCH 14/16] reftable/table: introduce iterator for table blocks","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-04-02T21:46:06Z","receivedAt":"2025-04-02T21:49:55Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/31 10:41AM, Patrick Steinhardt wrote:\n> Introduce a new iterator that allows the caller to iterate through all\n> blocks contained in a table. This gives users more fine-grained control\n> over how exactly those blocks are being read and exposes information to\n> callers that was previously inaccessible.\n> \n> This iterator will be required by a future patch series that adds\n> consistency checks for the reftable backend. In addition to that though\n> we will also reimplement `reftable_table_print_blocks()` on top of this\n> new iterator in a subsequent commit.\n> \n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  reftable/reftable-table.h       |  16 ++++++\n>  reftable/table.c                |  47 +++++++++++++++++\n>  t/unit-tests/t-reftable-table.c | 109 ++++++++++++++++++++++++++++++++++++++++\n>  3 files changed, 172 insertions(+)\n> \n> diff --git a/reftable/reftable-table.h b/reftable/reftable-table.h\n> index a78db9eea7e..f0f1784c664 100644\n> --- a/reftable/reftable-table.h\n> +++ b/reftable/reftable-table.h\n> @@ -10,6 +10,7 @@\n>  #define REFTABLE_TABLE_H\n>  \n>  #include \"reftable-iterator.h\"\n> +#include \"reftable-block.h\"\n>  #include \"reftable-blocksource.h\"\n>  \n>  /*\n> @@ -99,4 +100,19 @@ uint64_t reftable_table_min_update_index(struct reftable_table *t);\n>  /* print blocks onto stdout for debugging. */\n>  int reftable_table_print_blocks(const char *tablename);\n>  \n> +/*\n> + * An iterator that iterates through the blocks contained in a given table.\n> + */\n> +struct reftable_table_iterator {\n> +\tvoid *iter_arg;\n> +};\n> +\n> +int reftable_table_init_table_iterator(struct reftable_table *t,\n> +\t\t\t\t       struct reftable_table_iterator *it);\n\nAny reason we don't call this `reftable_table_iterator_init`? Seems like\nit would more closely match the other functions.\n\n> +\n> +void reftable_table_iterator_release(struct reftable_table_iterator *it);\n> +\n> +int reftable_table_iterator_next(struct reftable_table_iterator *it,\n> +\t\t\t\t const struct reftable_block **out);\n> +\n>  #endif\n"},{"id":"515542","messageId":"dlrvun53gnrd7lcwjp5m3q6q2gjjavdxnuhj4kie7hp2gfcskb@e7m3mnvk7rbu","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-16-ebed5247434c@pks.im","subject":"Re: [PATCH 16/16] reftable/table: move printing logic into test helper","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-04-02T21:52:32Z","receivedAt":"2025-04-02T21:56:22Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/31 10:41AM, Patrick Steinhardt wrote:\n> The logic to print individual blocks in a table is hosted in the\n> reftable library. This is only the case due to historical reasons though\n> because users of the library had no interfaces to read blocks one by\n> one. Otherwise, printing individual blocks has no place in the reftable\n> library given that the format will not be generic in the first place.\n> \n> We have now grown a public interface to iterate through blocks contained\n> in a table, and thus we can finally move the logic to print them into\n> the test helper.\n> \n> Move over the logic and refactor it accordingly. Note that the iterator\n> also trivially allows us to access index sections, which we previously\n> didn't print at all. This omission wasn't intentional though, so start\n> dumping those sections as well so that we can assert that indices are\n> written as expected.\n> \n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  reftable/reftable-table.h         |  3 --\n>  reftable/table.c                  | 65 ------------------------------------\n>  t/helper/test-reftable.c          | 69 ++++++++++++++++++++++++++++++++++++++-\n>  t/t0613-reftable-write-options.sh |  9 +++++\n>  4 files changed, 77 insertions(+), 69 deletions(-)\n> \n> diff --git a/reftable/reftable-table.h b/reftable/reftable-table.h\n> index f0f1784c664..293fffbddc6 100644\n> --- a/reftable/reftable-table.h\n> +++ b/reftable/reftable-table.h\n> @@ -97,9 +97,6 @@ uint64_t reftable_table_max_update_index(struct reftable_table *t);\n>  /* return the min_update_index for a table */\n>  uint64_t reftable_table_min_update_index(struct reftable_table *t);\n>  \n> -/* print blocks onto stdout for debugging. */\n> -int reftable_table_print_blocks(const char *tablename);\n> -\n>  /*\n>   * An iterator that iterates through the blocks contained in a given table.\n>   */\n> diff --git a/reftable/table.c b/reftable/table.c\n> index 48f0cdfd42b..8a7581b9800 100644\n> --- a/reftable/table.c\n> +++ b/reftable/table.c\n> @@ -740,71 +740,6 @@ uint64_t reftable_table_min_update_index(struct reftable_table *t)\n>  \treturn t->min_update_index;\n>  }\n>  \n> -int reftable_table_print_blocks(const char *tablename)\n> -{\n> -\tstruct {\n> -\t\tconst char *name;\n> -\t\tint type;\n> -\t} sections[] = {\n> -\t\t{\n> -\t\t\t.name = \"ref\",\n> -\t\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n> -\t\t},\n> -\t\t{\n> -\t\t\t.name = \"obj\",\n> -\t\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n> -\t\t},\n> -\t\t{\n> -\t\t\t.name = \"log\",\n> -\t\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n> -\t\t},\n> -\t};\n> -\tstruct reftable_block_source src = { 0 };\n> -\tstruct reftable_table *table = NULL;\n> -\tstruct table_iter ti = { 0 };\n> -\tsize_t i;\n> -\tint err;\n> -\n> -\terr = reftable_block_source_from_file(&src, tablename);\n> -\tif (err < 0)\n> -\t\tgoto done;\n> -\n> -\terr = reftable_table_new(&table, &src, tablename);\n> -\tif (err < 0)\n> -\t\tgoto done;\n> -\n> -\ttable_iter_init(&ti, table);\n> -\n> -\tprintf(\"header:\\n\");\n> -\tprintf(\"  block_size: %d\\n\", table->block_size);\n> -\n> -\tfor (i = 0; i < sizeof(sections) / sizeof(*sections); i++) {\n> -\t\terr = table_iter_seek_start(&ti, sections[i].type, 0);\n> -\t\tif (err < 0)\n> -\t\t\tgoto done;\n> -\t\tif (err > 0)\n> -\t\t\tcontinue;\n> -\n> -\t\tprintf(\"%s:\\n\", sections[i].name);\n> -\n> -\t\twhile (1) {\n> -\t\t\tprintf(\"  - length: %u\\n\", ti.block.restart_off);\n> -\t\t\tprintf(\"    restarts: %u\\n\", ti.block.restart_count);\n> -\n> -\t\t\terr = table_iter_next_block(&ti);\n> -\t\t\tif (err < 0)\n> -\t\t\t\tgoto done;\n> -\t\t\tif (err > 0)\n> -\t\t\t\tbreak;\n> -\t\t}\n> -\t}\n> -\n> -done:\n> -\treftable_table_decref(table);\n> -\ttable_iter_close(&ti);\n> -\treturn err;\n> -}\n\nHow that we have a table iterator exposed to iterate over the records we\ncan remove `reftable_table_print_blocks()` from the reftable library.\nMake sense.\n\n>  int reftable_table_init_table_iterator(struct reftable_table *t,\n>  \t\t\t\t       struct reftable_table_iterator *it)\n>  {\n> diff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c\n> index f8f1956f4f3..c465137826c 100644\n> --- a/t/helper/test-reftable.c\n> +++ b/t/helper/test-reftable.c\n> @@ -2,6 +2,7 @@\n>  #include \"hash.h\"\n>  #include \"hex.h\"\n>  #include \"reftable/system.h\"\n> +#include \"reftable/reftable-constants.h\"\n>  #include \"reftable/reftable-error.h\"\n>  #include \"reftable/reftable-merged.h\"\n>  #include \"reftable/reftable-stack.h\"\n> @@ -20,6 +21,72 @@ static void print_help(void)\n>  \t       \"\\n\");\n>  }\n>  \n> +static int dump_blocks(const char *tablename)\n> +{\n> +\tstruct reftable_table_iterator ti = { 0 };\n> +\tstruct reftable_block_source src = { 0 };\n> +\tstruct reftable_table *table = NULL;\n> +\tuint8_t section_type = 0;\n> +\tint err;\n> +\n> +\terr = reftable_block_source_from_file(&src, tablename);\n> +\tif (err < 0)\n> +\t\tgoto done;\n> +\n> +\terr = reftable_table_new(&table, &src, tablename);\n> +\tif (err < 0)\n> +\t\tgoto done;\n> +\n> +\terr = reftable_table_init_table_iterator(table, &ti);\n> +\tif (err < 0)\n> +\t\tgoto done;\n> +\n> +\tprintf(\"header:\\n\");\n> +\tprintf(\"  block_size: %d\\n\", table->block_size);\n> +\n> +\twhile (1) {\n> +\t\tconst struct reftable_block *block;\n> +\n> +\t\terr = reftable_table_iterator_next(&ti, &block);\n> +\t\tif (err < 0)\n> +\t\t\tgoto done;\n> +\t\tif (err > 0)\n> +\t\t\tbreak;\n> +\n> +\t\tif (block->block_type != section_type) {\n> +\t\t\tconst char *section;\n> +\t\t\tswitch (block->block_type) {\n> +\t\t\tcase REFTABLE_BLOCK_TYPE_LOG:\n> +\t\t\t\tsection = \"log\";\n> +\t\t\t\tbreak;\n> +\t\t\tcase REFTABLE_BLOCK_TYPE_REF:\n> +\t\t\t\tsection = \"ref\";\n> +\t\t\t\tbreak;\n> +\t\t\tcase REFTABLE_BLOCK_TYPE_OBJ:\n> +\t\t\t\tsection = \"obj\";\n> +\t\t\t\tbreak;\n> +\t\t\tcase REFTABLE_BLOCK_TYPE_INDEX:\n> +\t\t\t\tsection = \"idx\";\n> +\t\t\t\tbreak;\n> +\t\t\tdefault:\n> +\t\t\t\terr = -1;\n> +\t\t\t\tgoto done;\n> +\t\t\t}\n> +\n> +\t\t\tsection_type = block->block_type;\n> +\t\t\tprintf(\"%s:\\n\", section);\n> +\t\t}\n> +\n> +\t\tprintf(\"  - length: %u\\n\", block->restart_off);\n> +\t\tprintf(\"    restarts: %u\\n\", block->restart_count);\n> +\t}\n> +\n> +done:\n> +\treftable_table_iterator_release(&ti);\n> +\treftable_table_decref(table);\n> +\treturn err;\n> +}\n> +\n>  static int dump_table(struct reftable_merged_table *mt)\n>  {\n>  \tstruct reftable_iterator it = { NULL };\n> @@ -184,7 +251,7 @@ int cmd__dump_reftable(int argc, const char **argv)\n>  \targ = argv[1];\n>  \n>  \tif (opt_dump_blocks) {\n> -\t\terr = reftable_table_print_blocks(arg);\n> +\t\terr = dump_blocks(arg);\n\nNice!\n\n>  \t} else if (opt_dump_table) {\n>  \t\terr = dump_reftable(arg);\n>  \t} else if (opt_dump_stack) {\n> diff --git a/t/t0613-reftable-write-options.sh b/t/t0613-reftable-write-options.sh\n> index e2708e11d5b..e4c7461ce9e 100755\n> --- a/t/t0613-reftable-write-options.sh\n> +++ b/t/t0613-reftable-write-options.sh\n> @@ -93,6 +93,9 @@ test_expect_success 'many refs results in multiple blocks' '\n>  \t\t    restarts: 3\n>  \t\t  - length: 3289\n>  \t\t    restarts: 3\n> +\t\tidx:\n> +\t\t  - length: 103\n> +\t\t    restarts: 1\n>  \t\tEOF\n>  \t\ttest-tool dump-reftable -b .git/reftable/*.ref >actual &&\n>  \t\ttest_cmp expect actual\n> @@ -241,6 +244,9 @@ test_expect_success 'object index gets written by default with ref index' '\n>  \t\t    restarts: 1\n>  \t\t  - length: 80\n>  \t\t    restarts: 1\n> +\t\tidx:\n> +\t\t  - length: 55\n> +\t\t    restarts: 2\n>  \t\tobj:\n>  \t\t  - length: 11\n>  \t\t    restarts: 1\n> @@ -277,6 +283,9 @@ test_expect_success 'object index can be disabled' '\n>  \t\t    restarts: 1\n>  \t\t  - length: 80\n>  \t\t    restarts: 1\n> +\t\tidx:\n> +\t\t  - length: 55\n> +\t\t    restarts: 2\n>  \t\tEOF\n>  \t\ttest-tool dump-reftable -b .git/reftable/*.ref >actual &&\n>  \t\ttest_cmp expect actual\n> \n> -- \n> 2.49.0.604.gff1f9ca942.dirty\n> \n> \n"},{"id":"515578","messageId":"CAOLa=ZQV5ixN4TuM3KL=AuMYE=cW+KPqB7sLof7E4=F8Ep1Kag@mail.gmail.com","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-3-ebed5247434c@pks.im","subject":"Re: [PATCH 03/16] reftable/blocksource: consolidate code into a single file","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-03T10:42:42Z","receivedAt":"2025-04-03T10:42:44Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> The code that implements block sources is distributed across a couple of\n> files even though. Consolidate all of it into \"reftable/blocksource.c\"\n> and its accompanying header so that it is easier to locate and more self\n> contained.\n>\n\nSo 'reftable/blocksource.c' is an existing file and we're moving related\ncontent into this file. Makes sense.\n\n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  reftable/block.c                    | 17 +++--------------\n>  reftable/block.h                    |  3 ---\n>  reftable/blocksource.c              | 35 +++++++++++++++++++++++++++++++++++\n>  reftable/blocksource.h              | 27 ++++++++++++++++++++++++++-\n>  reftable/iter.c                     |  5 +++--\n>  reftable/reftable-blocksource.h     |  3 ++-\n>  reftable/table.c                    | 33 +++++----------------------------\n>  reftable/table.h                    |  7 -------\n>  t/unit-tests/t-reftable-block.c     |  8 ++++----\n>  t/unit-tests/t-reftable-readwrite.c |  4 ++--\n>  10 files changed, 80 insertions(+), 62 deletions(-)\n>\n> diff --git a/reftable/block.c b/reftable/block.c\n> index a5734d44415..97740187259 100644\n> --- a/reftable/block.c\n> +++ b/reftable/block.c\n> @@ -221,7 +221,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n>  \tuint32_t restart_start = 0;\n>  \tuint8_t *restart_bytes = NULL;\n>\n> -\treftable_block_done(&br->block);\n> +\tblock_source_return_block(&br->block);\n>\n>  \tif (!reftable_is_block_type(typ)) {\n>  \t\terr =  REFTABLE_FORMAT_ERROR;\n> @@ -285,7 +285,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n>  \t\t}\n>\n>  \t\t/* We're done with the input data. */\n> -\t\treftable_block_done(block);\n> +\t\tblock_source_return_block(block);\n>  \t\tblock->data = br->uncompressed_data;\n>  \t\tblock->len = sz;\n>  \t\tfull_block_size = src_len + block_header_skip - br->zstream->avail_in;\n> @@ -324,7 +324,7 @@ void block_reader_release(struct block_reader *br)\n>  \tinflateEnd(br->zstream);\n>  \treftable_free(br->zstream);\n>  \treftable_free(br->uncompressed_data);\n> -\treftable_block_done(&br->block);\n> +\tblock_source_return_block(&br->block);\n>  }\n>\n>  uint8_t block_reader_type(const struct block_reader *r)\n> @@ -570,14 +570,3 @@ void block_writer_release(struct block_writer *bw)\n>  \treftable_buf_release(&bw->last_key);\n>  \t/* the block is not owned. */\n>  }\n> -\n> -void reftable_block_done(struct reftable_block *blockp)\n> -{\n> -\tstruct reftable_block_source source = blockp->source;\n> -\tif (blockp && source.ops)\n> -\t\tsource.ops->return_block(source.arg, blockp);\n> -\tblockp->data = NULL;\n> -\tblockp->len = 0;\n> -\tblockp->source.ops = NULL;\n> -\tblockp->source.arg = NULL;\n> -}\n> diff --git a/reftable/block.h b/reftable/block.h\n> index eaeffdffc90..203b07d9a44 100644\n> --- a/reftable/block.h\n> +++ b/reftable/block.h\n> @@ -142,7 +142,4 @@ size_t header_size(int version);\n>  /* size of file footer, depending on format version */\n>  size_t footer_size(int version);\n>\n> -/* returns a block to its source. */\n> -void reftable_block_done(struct reftable_block *ret);\n> -\n>  #endif\n> diff --git a/reftable/blocksource.c b/reftable/blocksource.c\n> index 1397cbe7800..bc785506fb1 100644\n> --- a/reftable/blocksource.c\n> +++ b/reftable/blocksource.c\n> @@ -13,6 +13,41 @@\n>  #include \"reftable-blocksource.h\"\n>  #include \"reftable-error.h\"\n>\n> +void block_source_return_block(struct reftable_block *block)\n> +{\n> +\tstruct reftable_block_source source = block->source;\n> +\tif (block && source.ops)\n> +\t\tsource.ops->return_block(source.arg, block);\n> +\tblock->data = NULL;\n> +\tblock->len = 0;\n> +\tblock->source.ops = NULL;\n> +\tblock->source.arg = NULL;\n> +}\n> +\n> +void block_source_close(struct reftable_block_source *source)\n> +{\n> +\tif (!source->ops) {\n> +\t\treturn;\n> +\t}\n> +\n> +\tsource->ops->close(source->arg);\n> +\tsource->ops = NULL;\n> +}\n> +\n> +ssize_t block_source_read_block(struct reftable_block_source *source,\n> +\t\t\t\tstruct reftable_block *dest, uint64_t off,\n> +\t\t\t\tuint32_t size)\n> +{\n> +\tssize_t result = source->ops->read_block(source->arg, dest, off, size);\n> +\tdest->source = *source;\n> +\treturn result;\n> +}\n> +\n> +uint64_t block_source_size(struct reftable_block_source *source)\n> +{\n> +\treturn source->ops->size(source->arg);\n> +}\n> +\n>  static void reftable_buf_return_block(void *b REFTABLE_UNUSED, struct reftable_block *dest)\n>  {\n>  \tif (dest->len)\n> diff --git a/reftable/blocksource.h b/reftable/blocksource.h\n> index 7b67898ae22..639b9a1a3c5 100644\n> --- a/reftable/blocksource.h\n> +++ b/reftable/blocksource.h\n> @@ -12,9 +12,34 @@\n>  #include \"system.h\"\n>\n>  struct reftable_block_source;\n> +struct reftable_block;\n>  struct reftable_buf;\n>\n> -/* Create an in-memory block source for reading reftables */\n> +/*\n> + * Close the block source and the underlying resource. This is a no-op in case\n> + * the block source is zero-initialized.\n> + */\n> +void block_source_close(struct reftable_block_source *source);\n> +\n> +/*\n> + * Read a block of length `size` from the source at the given `off`.\n> + */\n> +ssize_t block_source_read_block(struct reftable_block_source *source,\n> +\t\t\t\tstruct reftable_block *dest, uint64_t off,\n> +\t\t\t\tuint32_t size);\n> +\n> +/*\n> + * Return the total length of the underlying resource.\n> + */\n> +uint64_t block_source_size(struct reftable_block_source *source);\n> +\n> +/*\n> + * Return a block to its original source, releasing any resources associated\n> + * with it.\n> + */\n> +void block_source_return_block(struct reftable_block *block);\n> +\n> +/* Create an in-memory block source for reading reftables. */\n>  void block_source_from_buf(struct reftable_block_source *bs,\n>  \t\t\t   struct reftable_buf *buf);\n>\n> diff --git a/reftable/iter.c b/reftable/iter.c\n> index 7376f263c99..6af6eb49396 100644\n> --- a/reftable/iter.c\n> +++ b/reftable/iter.c\n> @@ -11,6 +11,7 @@\n>  #include \"system.h\"\n>\n>  #include \"block.h\"\n> +#include \"blocksource.h\"\n>  #include \"constants.h\"\n>  #include \"reftable-error.h\"\n>  #include \"table.h\"\n> @@ -113,7 +114,7 @@ static void indexed_table_ref_iter_close(void *p)\n>  {\n>  \tstruct indexed_table_ref_iter *it = p;\n>  \tblock_iter_close(&it->cur);\n> -\treftable_block_done(&it->block_reader.block);\n> +\tblock_source_return_block(&it->block_reader.block);\n>  \treftable_free(it->offsets);\n>  \treftable_buf_release(&it->oid);\n>  }\n> @@ -127,7 +128,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n>  \t\treturn 1;\n>  \t}\n>\n> -\treftable_block_done(&it->block_reader.block);\n> +\tblock_source_return_block(&it->block_reader.block);\n>\n>  \toff = it->offsets[it->offset_idx++];\n>  \terr = table_init_block_reader(it->table, &it->block_reader, off,\n> diff --git a/reftable/reftable-blocksource.h b/reftable/reftable-blocksource.h\n> index 8692cd017e9..96430b629e4 100644\n> --- a/reftable/reftable-blocksource.h\n> +++ b/reftable/reftable-blocksource.h\n> @@ -11,7 +11,8 @@\n>\n>  #include <stdint.h>\n>\n> -/* block_source is a generic wrapper for a seekable readable file.\n> +/*\n> + * Generic wrapper for a seekable readable file.\n>   */\n>  struct reftable_block_source {\n>  \tstruct reftable_block_source_vtable *ops;\n> diff --git a/reftable/table.c b/reftable/table.c\n> index 440fb559ad1..d18e17b0d44 100644\n> --- a/reftable/table.c\n> +++ b/reftable/table.c\n> @@ -10,35 +10,12 @@\n>\n>  #include \"system.h\"\n>  #include \"block.h\"\n> +#include \"blocksource.h\"\n>  #include \"constants.h\"\n>  #include \"iter.h\"\n>  #include \"record.h\"\n>  #include \"reftable-error.h\"\n>\n> -uint64_t block_source_size(struct reftable_block_source *source)\n> -{\n> -\treturn source->ops->size(source->arg);\n> -}\n> -\n> -ssize_t block_source_read_block(struct reftable_block_source *source,\n> -\t\t\t\tstruct reftable_block *dest, uint64_t off,\n> -\t\t\t\tuint32_t size)\n> -{\n> -\tssize_t result = source->ops->read_block(source->arg, dest, off, size);\n> -\tdest->source = *source;\n> -\treturn result;\n> -}\n> -\n> -void block_source_close(struct reftable_block_source *source)\n> -{\n> -\tif (!source->ops) {\n> -\t\treturn;\n> -\t}\n> -\n> -\tsource->ops->close(source->arg);\n> -\tsource->ops = NULL;\n> -}\n> -\n>  static struct reftable_table_offsets *\n>  table_offsets_for(struct reftable_table *t, uint8_t typ)\n>  {\n> @@ -249,7 +226,7 @@ int table_init_block_reader(struct reftable_table *t, struct block_reader *br,\n>  \t}\n>\n>  \tif (block_size > guess_block_size) {\n> -\t\treftable_block_done(&block);\n> +\t\tblock_source_return_block(&block);\n>  \t\terr = table_get_block(t, &block, next_off, block_size);\n>  \t\tif (err < 0) {\n>  \t\t\tgoto done;\n> @@ -259,7 +236,7 @@ int table_init_block_reader(struct reftable_table *t, struct block_reader *br,\n>  \terr = block_reader_init(br, &block, header_off, t->block_size,\n>  \t\t\t\thash_size(t->hash_id));\n>  done:\n> -\treftable_block_done(&block);\n> +\tblock_source_return_block(&block);\n>\n>  \treturn err;\n>  }\n> @@ -666,8 +643,8 @@ int reftable_table_new(struct reftable_table **out,\n>  \t*out = t;\n>\n>  done:\n> -\treftable_block_done(&footer);\n> -\treftable_block_done(&header);\n> +\tblock_source_return_block(&footer);\n> +\tblock_source_return_block(&header);\n>  \tif (err) {\n>  \t\tif (t)\n>  \t\t\treftable_free(t->name);\n> diff --git a/reftable/table.h b/reftable/table.h\n> index 9cd8f80a207..8d8dd2b413d 100644\n> --- a/reftable/table.h\n> +++ b/reftable/table.h\n> @@ -14,13 +14,6 @@\n>  #include \"reftable-iterator.h\"\n>  #include \"reftable-table.h\"\n>\n> -uint64_t block_source_size(struct reftable_block_source *source);\n> -\n> -ssize_t block_source_read_block(struct reftable_block_source *source,\n> -\t\t\t\tstruct reftable_block *dest, uint64_t off,\n> -\t\t\t\tuint32_t size);\n> -void block_source_close(struct reftable_block_source *source);\n> -\n>  /* metadata for a block type */\n>  struct reftable_table_offsets {\n>  \tint is_present;\n> diff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c\n> index 22040aeefa5..8bb40482347 100644\n> --- a/t/unit-tests/t-reftable-block.c\n> +++ b/t/unit-tests/t-reftable-block.c\n> @@ -100,7 +100,7 @@ static void t_ref_block_read_write(void)\n>  \tblock_reader_release(&br);\n>  \tblock_iter_close(&it);\n>  \treftable_record_release(&rec);\n> -\treftable_block_done(&br.block);\n> +\tblock_source_return_block(&br.block);\n>  \treftable_buf_release(&want);\n>  \treftable_buf_release(&buf);\n>  \tfor (i = 0; i < N; i++)\n> @@ -190,7 +190,7 @@ static void t_log_block_read_write(void)\n>  \tblock_reader_release(&br);\n>  \tblock_iter_close(&it);\n>  \treftable_record_release(&rec);\n> -\treftable_block_done(&br.block);\n> +\tblock_source_return_block(&br.block);\n>  \treftable_buf_release(&want);\n>  \treftable_buf_release(&buf);\n>  \tfor (i = 0; i < N; i++)\n> @@ -273,7 +273,7 @@ static void t_obj_block_read_write(void)\n>  \tblock_reader_release(&br);\n>  \tblock_iter_close(&it);\n>  \treftable_record_release(&rec);\n> -\treftable_block_done(&br.block);\n> +\tblock_source_return_block(&br.block);\n>  \treftable_buf_release(&want);\n>  \treftable_buf_release(&buf);\n>  \tfor (i = 0; i < N; i++)\n> @@ -365,7 +365,7 @@ static void t_index_block_read_write(void)\n>  \tblock_reader_release(&br);\n>  \tblock_iter_close(&it);\n>  \treftable_record_release(&rec);\n> -\treftable_block_done(&br.block);\n> +\tblock_source_return_block(&br.block);\n>  \treftable_buf_release(&want);\n>  \treftable_buf_release(&buf);\n>  \tfor (i = 0; i < N; i++)\n> diff --git a/t/unit-tests/t-reftable-readwrite.c b/t/unit-tests/t-reftable-readwrite.c\n> index c4c27242ba9..3fba888cdaa 100644\n> --- a/t/unit-tests/t-reftable-readwrite.c\n> +++ b/t/unit-tests/t-reftable-readwrite.c\n> @@ -32,13 +32,13 @@ static void t_buffer(void)\n>  \tn = block_source_read_block(&source, &out, 0, sizeof(in));\n>  \tcheck_int(n, ==, sizeof(in));\n>  \tcheck(!memcmp(in, out.data, n));\n> -\treftable_block_done(&out);\n> +\tblock_source_return_block(&out);\n>\n>  \tn = block_source_read_block(&source, &out, 1, 2);\n>  \tcheck_int(n, ==, 2);\n>  \tcheck(!memcmp(out.data, \"el\", 2));\n>\n> -\treftable_block_done(&out);\n> +\tblock_source_return_block(&out);\n>  \tblock_source_close(&source);\n>  \treftable_buf_release(&buf);\n>  }\n>\n> --\n> 2.49.0.604.gff1f9ca942.dirty\n\nThe changes look good to me!\n"},{"id":"515590","messageId":"CAOLa=ZRwRkV56HAxtfX3EM1Lr3D938bY7d-zv+xUF4G40f-O2A@mail.gmail.com","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-4-ebed5247434c@pks.im","subject":"Re: [PATCH 04/16] reftable/block: simplify how we track restart points","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-03T15:17:50Z","receivedAt":"2025-04-03T15:17:52Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> Restart points record the location of reftable records that do not use\n> prefix compression and are used to perform a binary search inside of a\n> block. These restart points are encoded at the end of a block, between\n> the record data and the footer of a table.\n>\n> The block structure contains three different variables related to these\n> restart points:\n>\n>   - The block length contains the length of the reftable block up to the\n>     restart points.\n>\n>   - The restart count contains the number of restart points contained in\n>     the block.\n>\n>   - The restart bytes variable tracks where the restart point data\n>     begins.\n>\n> Tracking all three of these variables is unnecessary though as the data\n> can be derived from one another: the block length without restart points\n> is the exact same as the offset of the restart count data, which we\n> already track via the `restart_bytes` data.\n>\n\nNit: This para makes it seem as if we'd eliminate 'block length' in\nsupport of having/keeping `restart_bytes`, but we remove both.\n\n> Refactor the code so that we track the location of restart bytes not as\n> a pointer, but instead as an offset. This allows us to trivially get rid\n> of the `block_len` variable as described above. This avoids having the\n> confusing `block_len` variable and allows us to do less bookkeeping\n> overall.\n>\n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  reftable/block.c | 25 ++++++++++++-------------\n>  reftable/block.h |  8 +++++---\n>  reftable/table.c |  2 +-\n>  3 files changed, 18 insertions(+), 17 deletions(-)\n>\n> diff --git a/reftable/block.c b/reftable/block.c\n> index 97740187259..f2567a8f0fd 100644\n> --- a/reftable/block.c\n> +++ b/reftable/block.c\n> @@ -216,10 +216,9 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n>  \tuint32_t full_block_size = table_block_size;\n>  \tuint8_t typ = block->data[header_off];\n>  \tuint32_t sz = reftable_get_be24(block->data + header_off + 1);\n> -\tint err = 0;\n> -\tuint16_t restart_count = 0;\n> -\tuint32_t restart_start = 0;\n> -\tuint8_t *restart_bytes = NULL;\n> +\tuint16_t restart_count;\n> +\tuint32_t restart_off;\n\nNit: I guess this is to be consistent with `header_off`, but I would\nthink spelling it out as `header_offset` is much easier to understand.\n\n> +\tint err;\n>\n>  \tblock_source_return_block(&br->block);\n>\n> @@ -300,8 +299,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n>  \t}\n>\n>  \trestart_count = reftable_get_be16(block->data + sz - 2);\n> -\trestart_start = sz - 2 - 3 * restart_count;\n> -\trestart_bytes = block->data + restart_start;\n> +\trestart_off = sz - 2 - 3 * restart_count;\n>\n\nSo each block looks like:\n\n'r'\nuint24( block_len )\nref_record+\nuint24( restart_offset )+\nuint16( restart_count )\n\nSo this does make sense.\n\n>  \t/* transfer ownership. */\n>  \tbr->block = *block;\n> @@ -309,11 +307,12 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n>  \tblock->len = 0;\n>\n>  \tbr->hash_size = hash_size;\n> -\tbr->block_len = restart_start;\n> +\tbr->restart_off = restart_off;\n>  \tbr->full_block_size = full_block_size;\n>  \tbr->header_off = header_off;\n>  \tbr->restart_count = restart_count;\n> -\tbr->restart_bytes = restart_bytes;\n> +\n> +\terr = 0;\n>\n>  done:\n>  \treturn err;\n> @@ -337,7 +336,7 @@ int block_reader_first_key(const struct block_reader *br, struct reftable_buf *k\n>  \tint off = br->header_off + 4, n;\n>  \tstruct string_view in = {\n>  \t\t.buf = br->block.data + off,\n> -\t\t.len = br->block_len - off,\n> +\t\t.len = br->restart_off - off,\n>  \t};\n>  \tuint8_t extra = 0;\n>\n> @@ -354,13 +353,13 @@ int block_reader_first_key(const struct block_reader *br, struct reftable_buf *k\n>\n>  static uint32_t block_reader_restart_offset(const struct block_reader *br, size_t idx)\n>  {\n> -\treturn reftable_get_be24(br->restart_bytes + 3 * idx);\n> +\treturn reftable_get_be24(br->block.data + br->restart_off + 3 * idx);\n>  }\n>\n>  void block_iter_seek_start(struct block_iter *it, const struct block_reader *br)\n>  {\n>  \tit->block = br->block.data;\n> -\tit->block_len = br->block_len;\n> +\tit->block_len = br->restart_off;\n>  \tit->hash_size = br->hash_size;\n>  \treftable_buf_reset(&it->last_key);\n>  \tit->next_off = br->header_off + 4;\n> @@ -378,7 +377,7 @@ static int restart_needle_less(size_t idx, void *_args)\n>  \tuint32_t off = block_reader_restart_offset(args->reader, idx);\n>  \tstruct string_view in = {\n>  \t\t.buf = args->reader->block.data + off,\n> -\t\t.len = args->reader->block_len - off,\n> +\t\t.len = args->reader->restart_off - off,\n>  \t};\n>  \tuint64_t prefix_len, suffix_len;\n>  \tuint8_t extra;\n> @@ -505,7 +504,7 @@ int block_iter_seek_key(struct block_iter *it, const struct block_reader *br,\n>  \telse\n>  \t\tit->next_off = br->header_off + 4;\n>  \tit->block = br->block.data;\n> -\tit->block_len = br->block_len;\n> +\tit->block_len = br->restart_off;\n>  \tit->hash_size = br->hash_size;\n>\n>  \terr = reftable_record_init(&rec, block_reader_type(br));\n> diff --git a/reftable/block.h b/reftable/block.h\n> index 203b07d9a44..b78f322e646 100644\n> --- a/reftable/block.h\n> +++ b/reftable/block.h\n> @@ -79,10 +79,12 @@ struct block_reader {\n>  \tunsigned char *uncompressed_data;\n>  \tsize_t uncompressed_cap;\n>\n> -\t/* size of the data, excluding restart data. */\n> -\tuint32_t block_len;\n> -\tuint8_t *restart_bytes;\n> +\t/*\n> +\t * Restart point data. Restart points are located after the block's\n> +\t * record data.\n> +\t */\n>  \tuint16_t restart_count;\n> +\tuint32_t restart_off;\n>\n>  \t/* size of the data in the file. For log blocks, this is the compressed\n>  \t * size. */\n> diff --git a/reftable/table.c b/reftable/table.c\n> index d18e17b0d44..ec84545707c 100644\n> --- a/reftable/table.c\n> +++ b/reftable/table.c\n> @@ -838,7 +838,7 @@ int reftable_table_print_blocks(const char *tablename)\n>  \t\tprintf(\"%s:\\n\", sections[i].name);\n>\n>  \t\twhile (1) {\n> -\t\t\tprintf(\"  - length: %u\\n\", ti.br.block_len);\n> +\t\t\tprintf(\"  - length: %u\\n\", ti.br.restart_off);\n>  \t\t\tprintf(\"    restarts: %u\\n\", ti.br.restart_count);\n>\n>  \t\t\terr = table_iter_next_block(&ti);\n>\n> --\n> 2.49.0.604.gff1f9ca942.dirty\n\nThe patch looks good.\n"},{"id":"515755","messageId":"Z_PFfNwJVfVIuTPO@pks.im","threadId":"63222","inReplyTo":"y47rmb3ahghwvhdukfj27fihddmg57rqpfcqmaylfejqjjxvrr@luc7xuu5qiza","subject":"Re: [PATCH 07/16] reftable/block: rename `block_reader` to `reftable_block`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T12:30:52Z","receivedAt":"2025-04-07T12:30:57Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Apr 02, 2025 at 03:39:57PM -0500, Justin Tobler wrote:\n> On 25/03/31 10:41AM, Patrick Steinhardt wrote:\n> > diff --git a/reftable/block.c b/reftable/block.c\n> > index ad162ecdbf6..d188665388d 100644\n> > --- a/reftable/block.c\n> > +++ b/reftable/block.c\n> > @@ -222,10 +222,10 @@ static int read_block(struct reftable_block_source *source,\n> >  \treturn block_source_read_data(source, dest, off, sz);\n> >  }\n> >  \n> > -int block_reader_init(struct block_reader *br,\n> > -\t\t      struct reftable_block_source *source,\n> > -\t\t      uint32_t offset, uint32_t header_size,\n> > -\t\t      uint32_t table_block_size, uint32_t hash_size)\n> > +int reftable_block_init(struct reftable_block *block,\n> > +\t\t\tstruct reftable_block_source *source,\n> > +\t\t\tuint32_t offset, uint32_t header_size,\n> > +\t\t\tuint32_t table_block_size, uint32_t hash_size)\n> >  {\n> >  \tuint32_t guess_block_size = table_block_size ?\n> >  \t\ttable_block_size : DEFAULT_BLOCK_SIZE;\n> > @@ -236,19 +236,19 @@ int block_reader_init(struct block_reader *br,\n> >  \tuint8_t block_type;\n> >  \tint err;\n> >  \n> > -\terr = read_block(source, &br->block, offset, guess_block_size);\n> > +\terr = read_block(source, &block->block, offset, guess_block_size);\n> >  \tif (err < 0)\n> >  \t\tgoto done;\n> >  \n> > -\tblock_type = br->block.data[header_size];\n> > +\tblock_type = block->block.data[header_size];\n> \n> Ah, in the previous patch I suggested renaming the `block` field to\n> `data`, but I guess that would lead to `block->data.data` here and would\n> also be a bit funny. It's probably fine to just leave it as-is then.\n\nFair, but I think if we rename this to `block_data` then it's fine and\nhelps to clarify things a bit. I've done that now.\n\nPatrick\n"},{"id":"515756","messageId":"Z_PFfwf4D8HtYr_a@pks.im","threadId":"63222","inReplyTo":"y3lbo4dnuv46kgy7fhzcksdea2pdm6nsy43tjai6tcgd6b3w5u@w5zqty5tgpmt","subject":"Re: [PATCH 06/16] reftable/block: rename `block` to `block_data`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T12:30:55Z","receivedAt":"2025-04-07T12:30:58Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Apr 02, 2025 at 03:26:26PM -0500, Justin Tobler wrote:\n> On 25/03/31 10:41AM, Patrick Steinhardt wrote:\n> > The `reftable_block` structure associates a byte slice with a block\n> > source. As such it only holds the data of a reftable block without\n> > actually encoding any of the details for how to access that data.\n> > \n> > Rename the structure to instead be called `reftable_block_data`. Besides\n> > clarifying that this really only holds data, it also allows us to rename\n> > the `reftable_block_reader` to `reftable_block` in the next commit, as\n> > this is the structure that actually encapsulates access to the reftable\n> > blocks.\n> > \n> > Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> > ---\n> >  reftable/block.c                    | 10 ++++-----\n> >  reftable/block.h                    |  2 +-\n> >  reftable/blocksource.c              | 44 ++++++++++++++++++-------------------\n> >  reftable/blocksource.h              | 10 ++++-----\n> >  reftable/iter.c                     |  4 ++--\n> >  reftable/reftable-blocksource.h     | 14 ++++++------\n> >  reftable/table.c                    | 14 ++++++------\n> >  t/unit-tests/t-reftable-readwrite.c | 10 ++++-----\n> >  8 files changed, 54 insertions(+), 54 deletions(-)\n> [snip]  \n> > diff --git a/reftable/block.h b/reftable/block.h\n> > index 6afb1b2952a..c55d47c3c52 100644\n> > --- a/reftable/block.h\n> > +++ b/reftable/block.h\n> > @@ -71,7 +71,7 @@ struct block_reader {\n> >  \tuint32_t header_off;\n> >  \n> >  \t/* the memory block */\n> > -\tstruct reftable_block block;\n> > +\tstruct reftable_block_data block;\n> \n> I wonder if we should rename the `reftable_block_data` here from `block`\n> to `data`. In the next commit we rename `block_reader` to `block` which\n> leads to `block->block` in some places which I think looks a bit funny.\n\nFair. Will rename it to `block_data` though to match the struct name.\n\nPatrick\n"},{"id":"515757","messageId":"Z_PFgoVmcv-B1V7L@pks.im","threadId":"63222","inReplyTo":"5dbbo5wiszgx74ijinu7xkf7ptddoubkzt53sc6cg5ak357fja@x7v2kra4ny32","subject":"Re: [PATCH 11/16] reftable/block: make block iterators reseekable","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T12:30:58Z","receivedAt":"2025-04-07T12:31:01Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Apr 02, 2025 at 04:24:53PM -0500, Justin Tobler wrote:\n> On 25/03/31 10:41AM, Patrick Steinhardt wrote:\n> > diff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c\n> > index e36ed7ac576..c4ced39a73b 100644\n> > --- a/t/unit-tests/t-reftable-block.c\n> > +++ b/t/unit-tests/t-reftable-block.c\n> > @@ -66,7 +66,8 @@ static void t_ref_block_read_write(void)\n> >  \tblock_source_from_buf(&source ,&block_data);\n> >  \treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n> >  \n> > -\tblock_iter_seek_start(&it, &block);\n> > +\tblock_iter_init(&it, &block);\n> > +\tblock_iter_seek_start(&it);\n> \n> If I'm understanding correctly, `block_iter_init()` already invokes\n> `block_iter_seek_start()`. Why do we need to invoke\n> `block_iter_seek_start()` again?\n\nAh, good catch, we don't. Will simplify.\n\nPatrick\n"},{"id":"515758","messageId":"Z_PFiKs9s3R9pVW6@pks.im","threadId":"63222","inReplyTo":"u4h53ukh5uxk4r4zy5iyniob4xf7qmtem7j2kp45sknfbljwuv@ntdojrh6aakb","subject":"Re: [PATCH 10/16] reftable/block: store block pointer in the block iterator","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T12:31:04Z","receivedAt":"2025-04-07T12:31:07Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Apr 02, 2025 at 03:56:30PM -0500, Justin Tobler wrote:\n> On 25/03/31 10:41AM, Patrick Steinhardt wrote:\n> > The block iterator requires access to a bunch of data from the\n> > underlying `reftable_block` that it is iterating over. This data is\n> > stored by copying over relevant data into a separate set of variables.\n> > This has multiple downsides:\n> > \n> >   - We require more storage space than necessary. This is more of a\n> >     theoretical issue as we shouldn't ever have many blocks.\n> > \n> >   - We have to perform more bookkeeping, and the variable names are\n> >     inconsistent across the two data structures. This can lead to some\n> >     confusion.\n> > \n> >   - The lifetime of the block iterator is tied to the block anyway, but\n> >     we hide that a bit by only storing pointers into the block.\n> \n> s/into/in/\n\nI think \"into\" is right. What I want to say is that the pointers point\ninto the block itself, not that we're storing pointers in the block.\nI'll add \"pointing\" to clarify.\n\nPatrick\n"},{"id":"515759","messageId":"Z_PFi4-B3ACD5hmq@pks.im","threadId":"63222","inReplyTo":"CAOLa=ZRwRkV56HAxtfX3EM1Lr3D938bY7d-zv+xUF4G40f-O2A@mail.gmail.com","subject":"Re: [PATCH 04/16] reftable/block: simplify how we track restart points","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T12:31:07Z","receivedAt":"2025-04-07T12:31:10Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Apr 03, 2025 at 08:17:50AM -0700, Karthik Nayak wrote:\n> Patrick Steinhardt <ps@pks.im> writes:\n> \n> > Restart points record the location of reftable records that do not use\n> > prefix compression and are used to perform a binary search inside of a\n> > block. These restart points are encoded at the end of a block, between\n> > the record data and the footer of a table.\n> >\n> > The block structure contains three different variables related to these\n> > restart points:\n> >\n> >   - The block length contains the length of the reftable block up to the\n> >     restart points.\n> >\n> >   - The restart count contains the number of restart points contained in\n> >     the block.\n> >\n> >   - The restart bytes variable tracks where the restart point data\n> >     begins.\n> >\n> > Tracking all three of these variables is unnecessary though as the data\n> > can be derived from one another: the block length without restart points\n> > is the exact same as the offset of the restart count data, which we\n> > already track via the `restart_bytes` data.\n> >\n> \n> Nit: This para makes it seem as if we'd eliminate 'block length' in\n> support of having/keeping `restart_bytes`, but we remove both.\n\nWe don't, we only remove the block length. The restart bytes are\nretained, but they are renamed to `restart_off` to better reflect what\nit actually contains. The next paragraph tries to explain this:\n\n> > Refactor the code so that we track the location of restart bytes not as\n> > a pointer, but instead as an offset. This allows us to trivially get rid\n> > of the `block_len` variable as described above. This avoids having the\n> > confusing `block_len` variable and allows us to do less bookkeeping\n> > overall.\n> >\n> > Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> > ---\n> >  reftable/block.c | 25 ++++++++++++-------------\n> >  reftable/block.h |  8 +++++---\n> >  reftable/table.c |  2 +-\n> >  3 files changed, 18 insertions(+), 17 deletions(-)\n> >\n> > diff --git a/reftable/block.c b/reftable/block.c\n> > index 97740187259..f2567a8f0fd 100644\n> > --- a/reftable/block.c\n> > +++ b/reftable/block.c\n> > @@ -216,10 +216,9 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n> >  \tuint32_t full_block_size = table_block_size;\n> >  \tuint8_t typ = block->data[header_off];\n> >  \tuint32_t sz = reftable_get_be24(block->data + header_off + 1);\n> > -\tint err = 0;\n> > -\tuint16_t restart_count = 0;\n> > -\tuint32_t restart_start = 0;\n> > -\tuint8_t *restart_bytes = NULL;\n> > +\tuint16_t restart_count;\n> > +\tuint32_t restart_off;\n> \n> Nit: I guess this is to be consistent with `header_off`, but I would\n> think spelling it out as `header_offset` is much easier to understand.\n\nYeah, I'm not much of a fan of such abbreviations, either. But I'd like\nto retain this abbreviation for the sake of consistency.\n\nPatrick\n"},{"id":"515760","messageId":"Z_PFjr9p4u4MA4yD@pks.im","threadId":"63222","inReplyTo":"umaolgemnks4g4cgwgnurl7alic2a3pzrfx2ws3s7uxypiedia@7mwqxvhw2zxg","subject":"Re: [PATCH 05/16] reftable/table: move reading block into block reader","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T12:31:10Z","receivedAt":"2025-04-07T12:31:14Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Apr 02, 2025 at 03:13:03PM -0500, Justin Tobler wrote:\n> On 25/03/31 10:41AM, Patrick Steinhardt wrote:\n> > diff --git a/reftable/block.c b/reftable/block.c\n> > index f2567a8f0fd..2517108b8ef 100644\n> > --- a/reftable/block.c\n> > +++ b/reftable/block.c\n> > @@ -209,31 +209,57 @@ int block_writer_finish(struct block_writer *w)\n> >  \treturn w->next;\n> >  }\n> >  \n> > -int block_reader_init(struct block_reader *br, struct reftable_block *block,\n> > -\t\t      uint32_t header_off, uint32_t table_block_size,\n> > -\t\t      uint32_t hash_size)\n> > +static int read_block(struct reftable_block_source *source,\n> > +\t\t      struct reftable_block *dest, uint64_t off,\n> > +\t\t      uint32_t sz)\n> >  {\n> > +\tsize_t size = block_source_size(source);\n> > +\tblock_source_return_block(dest);\n> > +\tif (off >= size)\n> > +\t\treturn 0;\n> > +\tif (off + sz > size)\n> > +\t\tsz = size - off;\n> > +\treturn block_source_read_block(source, dest, off, sz);\n> > +}\n> > +\n> > +int block_reader_init(struct block_reader *br,\n> > +\t\t      struct reftable_block_source *source,\n> > +\t\t      uint32_t offset, uint32_t header_size,\n> > +\t\t      uint32_t table_block_size, uint32_t hash_size)\n> > +{\n> > +\tuint32_t guess_block_size = table_block_size ?\n> > +\t\ttable_block_size : DEFAULT_BLOCK_SIZE;\n> \n> Out of curiousity, in what scenarios would the table not know the block\n> size and we have to rely on the guess?\n\nBy default, reftable blocks are aligned with padding, and if so the\nblock size is tracked as part of the footer and well-known. Ideally, the\nblock size would be picked so that it is the same as your disk sector\nsize. Git picks 4kB by default.\n\nThere is another mode though: reftable blocks can be unaligned, so the\npadding is dropped. Consequently there is no fixed block siz, and this\nis indicated by having a block size of 0 in the header/trailer.\n\nPatrick\n"},{"id":"515761","messageId":"Z_PFkoNgCXrqZUwl@pks.im","threadId":"63222","inReplyTo":"q6shqgjoe5vlt3fduj6cq6473plz7mmlmnmyuwft7huuod5hhn@je7sijo3v54z","subject":"Re: [PATCH 14/16] reftable/table: introduce iterator for table blocks","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T12:31:14Z","receivedAt":"2025-04-07T12:31:18Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Apr 02, 2025 at 04:46:06PM -0500, Justin Tobler wrote:\n> On 25/03/31 10:41AM, Patrick Steinhardt wrote:\n> > diff --git a/reftable/reftable-table.h b/reftable/reftable-table.h\n> > index a78db9eea7e..f0f1784c664 100644\n> > --- a/reftable/reftable-table.h\n> > +++ b/reftable/reftable-table.h\n> > @@ -10,6 +10,7 @@\n> >  #define REFTABLE_TABLE_H\n> >  \n> >  #include \"reftable-iterator.h\"\n> > +#include \"reftable-block.h\"\n> >  #include \"reftable-blocksource.h\"\n> >  \n> >  /*\n> > @@ -99,4 +100,19 @@ uint64_t reftable_table_min_update_index(struct reftable_table *t);\n> >  /* print blocks onto stdout for debugging. */\n> >  int reftable_table_print_blocks(const char *tablename);\n> >  \n> > +/*\n> > + * An iterator that iterates through the blocks contained in a given table.\n> > + */\n> > +struct reftable_table_iterator {\n> > +\tvoid *iter_arg;\n> > +};\n> > +\n> > +int reftable_table_init_table_iterator(struct reftable_table *t,\n> > +\t\t\t\t       struct reftable_table_iterator *it);\n> \n> Any reason we don't call this `reftable_table_iterator_init`? Seems like\n> it would more closely match the other functions.\n\nNo, good point. We should change the parameter order in that case, as\nwell.\n\nPatrick\n"},{"id":"515763","messageId":"20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im","threadId":"63222","inReplyTo":"20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im","subject":"[PATCH v2 00/16] reftable: overhaul the API to expose access to blocks","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T13:16:12Z","receivedAt":"2025-04-07T13:16:24Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nthis patch series is a bigger overhaul of the reftable API. This\noverhaul has two main motivations:\n\n  - The reftable library is now standalone and can be used by code bases\n    other than Git, like libgit2. This patch series thus renames a\n    couple of subsystems to have more intuitive names before we gain any\n    new users.\n\n  - Some of the data of reftables isn't accessible at all via public\n    interfaces. Most importantly, it is not possible to access\n    individual blocks in a table. While users shouldn't need that access\n    most of the time, an upcoming usecase that Git itself has is to\n    implement consistency checks for the reftable backend. Here we'll\n    want to read through blocks and their respective records one by one\n    to ensure that they are sane and then iterate through records\n    contained in these blocks.\n\nThe patch series is structured as follows:\n\n  - Patch 1 is a trivial global refactoring to fix formatting of the\n    license headers. They have been annoying me for far too long.\n\n  - Patches 2 to 8 consolidate and rename a couple of data structures:\n\n      - `reftable_reader` becomes `reftable_table`, as it is used to\n        access an individual table.\n\n      - `reftable_block` becomes `reftable_block_data`, as it is only a\n        simple container for the underlying bytes.\n\n      - `reftable_block_reader` becomes `reftable_block`, as it is used\n        to access an individual block.\n\n    Overall, the data structures are now called after what they provide\n    access to compared to the rather generic previous names. This is\n    also in line with other data structures like `reftable_merged_table`\n    and `reftable_stack`.\n\n  - Patches 9 to 13 refactor the block interface so that it can expose a\n    generic `reftable_iterator`, granting generic access to all of its\n    contained records.\n\n  - Patches 14 to 16 refactor the table interface to expose a new\n    iterator over its contained blocks.\n\n  - Patch 17 refactors `reftable_table_print_blocks` to be implemented\n    on top of these new iterators. This allows us to move it out of the\n    library codebase into the test helper.\n\nThe series is built on Git v2.49.0 with ps/reftable-sans-compat-util at\n8f6a2dbe340 (Makefile: skip reftable library for Coccinelle, 2025-02-18)\nmerged into it.\n\nChanges in v2:\n  - Rename `reftable_table_init_table_iterator()` to\n    `reftable_table_iterator_init()`.\n  - Rename `struct reftable_block_reader::block` to `block_data`.\n  - Several improvements to commit messages.\n  - Drop unneeded calls to `block_iter_seek_start()` after\n    `block_iter_init()`.\n  - Fix -Wcomma error.\n  - Link to v1: https://lore.kernel.org/r/20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im\n\nThanks!\n\nPatrick\n\n---\nPatrick Steinhardt (16):\n      reftable: fix formatting of the license header\n      reftable/reader: rename data structure to \"table\"\n      reftable/blocksource: consolidate code into a single file\n      reftable/block: simplify how we track restart points\n      reftable/table: move reading block into block reader\n      reftable/block: rename `block` to `block_data`\n      reftable/block: rename `block_reader` to `reftable_block`\n      git-zlib: use `struct z_stream_s` instead of typedef\n      reftable/block: create public interface for reading blocks\n      reftable/block: store block pointer in the block iterator\n      reftable/block: make block iterators reseekable\n      reftable/block: expose a generic iterator over reftable records\n      reftable/table: add `reftable_table` to the public interface\n      reftable/table: introduce iterator for table blocks\n      reftable/constants: make block types part of the public interface\n      reftable/table: move printing logic into test helper\n\n .../howto/recover-corrupted-object-harder.adoc     |   4 +-\n Makefile                                           |   4 +-\n compat/zlib-compat.h                               |   4 +-\n git-zlib.h                                         |   2 +-\n meson.build                                        |   2 +-\n reftable/basics.c                                  |  12 +-\n reftable/basics.h                                  |  19 +-\n reftable/block.c                                   | 284 ++++++++-----\n reftable/block.h                                   |  85 ++--\n reftable/blocksource.c                             |  67 ++-\n reftable/blocksource.h                             |  39 +-\n reftable/constants.h                               |  18 +-\n reftable/error.c                                   |  12 +-\n reftable/iter.c                                    |  36 +-\n reftable/iter.h                                    |  18 +-\n reftable/merged.c                                  |  42 +-\n reftable/merged.h                                  |  16 +-\n reftable/pq.c                                      |  12 +-\n reftable/pq.h                                      |  12 +-\n reftable/reader.h                                  |  67 ---\n reftable/record.c                                  |  52 +--\n reftable/record.h                                  |  12 +-\n reftable/reftable-basics.h                         |  10 +-\n reftable/reftable-block.h                          |  74 ++++\n reftable/reftable-blocksource.h                    |  29 +-\n reftable/reftable-constants.h                      |  18 +\n reftable/reftable-error.h                          |  12 +-\n reftable/reftable-iterator.h                       |  12 +-\n reftable/reftable-merged.h                         |  18 +-\n reftable/reftable-reader.h                         |  72 ----\n reftable/reftable-record.h                         |  12 +-\n reftable/reftable-stack.h                          |  12 +-\n reftable/reftable-table.h                          | 115 +++++\n reftable/reftable-writer.h                         |  12 +-\n reftable/stack.c                                   | 188 ++++-----\n reftable/stack.h                                   |  16 +-\n reftable/system.h                                  |  12 +-\n reftable/{reader.c => table.c}                     | 463 +++++++++------------\n reftable/table.h                                   |  29 ++\n reftable/tree.c                                    |  12 +-\n reftable/tree.h                                    |  12 +-\n reftable/writer.c                                  |  34 +-\n reftable/writer.h                                  |  12 +-\n t/helper/test-reftable.c                           |  81 +++-\n t/meson.build                                      |   2 +-\n t/t0613-reftable-write-options.sh                  |   9 +\n t/unit-tests/t-reftable-block.c                    | 214 +++++++---\n t/unit-tests/t-reftable-merged.c                   |  86 ++--\n t/unit-tests/t-reftable-pq.c                       |  10 +-\n t/unit-tests/t-reftable-reader.c                   |  96 -----\n t/unit-tests/t-reftable-readwrite.c                | 106 ++---\n t/unit-tests/t-reftable-record.c                   |  40 +-\n t/unit-tests/t-reftable-stack.c                    |  66 +--\n t/unit-tests/t-reftable-table.c                    | 206 +++++++++\n 54 files changed, 1635 insertions(+), 1274 deletions(-)\n\nRange-diff versus v1:\n\n 1:  a02dacaad85 =  1:  0647a1770ac reftable: fix formatting of the license header\n 2:  29d3e212de1 =  2:  0729191d831 reftable/reader: rename data structure to \"table\"\n 3:  0140ff1c37b !  3:  64f35be377e reftable/blocksource: consolidate code into a single file\n    @@ Commit message\n         reftable/blocksource: consolidate code into a single file\n     \n         The code that implements block sources is distributed across a couple of\n    -    files even though. Consolidate all of it into \"reftable/blocksource.c\"\n    -    and its accompanying header so that it is easier to locate and more self\n    +    files. Consolidate all of it into \"reftable/blocksource.c\" and its\n    +    accompanying header so that it is easier to locate and more self\n         contained.\n     \n    +    While at it, rename some of the functions to have properly scoped names.\n    +\n         Signed-off-by: Patrick Steinhardt <ps@pks.im>\n     \n      ## reftable/block.c ##\n 4:  f735578f50f =  4:  ad57660df13 reftable/block: simplify how we track restart points\n 5:  dbce90b9520 !  5:  0ac16513e5c reftable/table: move reading block into block reader\n    @@ Commit message\n     \n         Refactor the code so that we stop passing the block when initializing a\n         reader, but instead by passing in the block source plus the offset at\n    -    which we we're supposed to read a block. Like this, the ownership of the\n    +    which we're supposed to read a block. Like this, the ownership of the\n         block itself doesn't need to get handed over as the block reader is the\n         one owning the block right from the start.\n     \n 6:  9a18a37e236 !  6:  30b4b1f37ed reftable/block: rename `block` to `block_data`\n    @@ Commit message\n         this is the structure that actually encapsulates access to the reftable\n         blocks.\n     \n    +    Rename the `struct reftable_block_reader::block` member accordingly.\n    +\n         Signed-off-by: Patrick Steinhardt <ps@pks.im>\n     \n      ## reftable/block.c ##\n    @@ reftable/block.c: int block_writer_finish(struct block_writer *w)\n      }\n      \n      int block_reader_init(struct block_reader *br,\n    +@@ reftable/block.c: int block_reader_init(struct block_reader *br,\n    + \tuint8_t block_type;\n    + \tint err;\n    + \n    +-\terr = read_block(source, &br->block, offset, guess_block_size);\n    ++\terr = read_block(source, &br->block_data, offset, guess_block_size);\n    + \tif (err < 0)\n    + \t\tgoto done;\n    + \n    +-\tblock_type = br->block.data[header_size];\n    ++\tblock_type = br->block_data.data[header_size];\n    + \tif (!reftable_is_block_type(block_type)) {\n    + \t\terr = REFTABLE_FORMAT_ERROR;\n    + \t\tgoto done;\n    + \t}\n    + \n    +-\tblock_size = reftable_get_be24(br->block.data + header_size + 1);\n    ++\tblock_size = reftable_get_be24(br->block_data.data + header_size + 1);\n    + \tif (block_size > guess_block_size) {\n    +-\t\terr = read_block(source, &br->block, offset, block_size);\n    ++\t\terr = read_block(source, &br->block_data, offset, block_size);\n    + \t\tif (err < 0)\n    + \t\t\tgoto done;\n    + \t}\n    +@@ reftable/block.c: int block_reader_init(struct block_reader *br,\n    + \tif (block_type == BLOCK_TYPE_LOG) {\n    + \t\tuint32_t block_header_skip = 4 + header_size;\n    + \t\tuLong dst_len = block_size - block_header_skip;\n    +-\t\tuLong src_len = br->block.len - block_header_skip;\n    ++\t\tuLong src_len = br->block_data.len - block_header_skip;\n    + \n    + \t\t/* Log blocks specify the *uncompressed* size in their header. */\n    + \t\tREFTABLE_ALLOC_GROW_OR_NULL(br->uncompressed_data, block_size,\n    +@@ reftable/block.c: int block_reader_init(struct block_reader *br,\n    + \t\t}\n    + \n    + \t\t/* Copy over the block header verbatim. It's not compressed. */\n    +-\t\tmemcpy(br->uncompressed_data, br->block.data, block_header_skip);\n    ++\t\tmemcpy(br->uncompressed_data, br->block_data.data, block_header_skip);\n    + \n    + \t\tif (!br->zstream) {\n    + \t\t\tREFTABLE_CALLOC_ARRAY(br->zstream, 1);\n    +@@ reftable/block.c: int block_reader_init(struct block_reader *br,\n    + \t\t\tgoto done;\n    + \t\t}\n    + \n    +-\t\tbr->zstream->next_in = br->block.data + block_header_skip;\n    ++\t\tbr->zstream->next_in = br->block_data.data + block_header_skip;\n    + \t\tbr->zstream->avail_in = src_len;\n    + \t\tbr->zstream->next_out = br->uncompressed_data + block_header_skip;\n    + \t\tbr->zstream->avail_out = dst_len;\n     @@ reftable/block.c: int block_reader_init(struct block_reader *br,\n      \t\t}\n      \n      \t\t/* We're done with the input data. */\n     -\t\tblock_source_return_block(&br->block);\n    -+\t\tblock_source_release_data(&br->block);\n    - \t\tbr->block.data = br->uncompressed_data;\n    - \t\tbr->block.len = block_size;\n    +-\t\tbr->block.data = br->uncompressed_data;\n    +-\t\tbr->block.len = block_size;\n    ++\t\tblock_source_release_data(&br->block_data);\n    ++\t\tbr->block_data.data = br->uncompressed_data;\n    ++\t\tbr->block_data.len = block_size;\n      \t\tfull_block_size = src_len + block_header_skip - br->zstream->avail_in;\n    + \t} else if (full_block_size == 0) {\n    + \t\tfull_block_size = block_size;\n    +-\t} else if (block_size < full_block_size && block_size < br->block.len &&\n    +-\t\t   br->block.data[block_size] != 0) {\n    ++\t} else if (block_size < full_block_size && block_size < br->block_data.len &&\n    ++\t\t   br->block_data.data[block_size] != 0) {\n    + \t\t/* If the block is smaller than the full block size, it is\n    + \t\t   padded (data followed by '\\0') or the next block is\n    + \t\t   unaligned. */\n    + \t\tfull_block_size = block_size;\n    + \t}\n    + \n    +-\trestart_count = reftable_get_be16(br->block.data + block_size - 2);\n    ++\trestart_count = reftable_get_be16(br->block_data.data + block_size - 2);\n    + \trestart_off = block_size - 2 - 3 * restart_count;\n    + \n    + \tbr->block_type = block_type;\n     @@ reftable/block.c: void block_reader_release(struct block_reader *br)\n      \tinflateEnd(br->zstream);\n      \treftable_free(br->zstream);\n      \treftable_free(br->uncompressed_data);\n     -\tblock_source_return_block(&br->block);\n    -+\tblock_source_release_data(&br->block);\n    ++\tblock_source_release_data(&br->block_data);\n      \tmemset(br, 0, sizeof(*br));\n      }\n      \n    + uint8_t block_reader_type(const struct block_reader *r)\n    + {\n    +-\treturn r->block.data[r->header_off];\n    ++\treturn r->block_data.data[r->header_off];\n    + }\n    + \n    + int block_reader_first_key(const struct block_reader *br, struct reftable_buf *key)\n    + {\n    + \tint off = br->header_off + 4, n;\n    + \tstruct string_view in = {\n    +-\t\t.buf = br->block.data + off,\n    ++\t\t.buf = br->block_data.data + off,\n    + \t\t.len = br->restart_off - off,\n    + \t};\n    + \tuint8_t extra = 0;\n    +@@ reftable/block.c: int block_reader_first_key(const struct block_reader *br, struct reftable_buf *k\n    + \n    + static uint32_t block_reader_restart_offset(const struct block_reader *br, size_t idx)\n    + {\n    +-\treturn reftable_get_be24(br->block.data + br->restart_off + 3 * idx);\n    ++\treturn reftable_get_be24(br->block_data.data + br->restart_off + 3 * idx);\n    + }\n    + \n    + void block_iter_seek_start(struct block_iter *it, const struct block_reader *br)\n    + {\n    +-\tit->block = br->block.data;\n    ++\tit->block = br->block_data.data;\n    + \tit->block_len = br->restart_off;\n    + \tit->hash_size = br->hash_size;\n    + \treftable_buf_reset(&it->last_key);\n    +@@ reftable/block.c: static int restart_needle_less(size_t idx, void *_args)\n    + \tstruct restart_needle_less_args *args = _args;\n    + \tuint32_t off = block_reader_restart_offset(args->reader, idx);\n    + \tstruct string_view in = {\n    +-\t\t.buf = args->reader->block.data + off,\n    ++\t\t.buf = args->reader->block_data.data + off,\n    + \t\t.len = args->reader->restart_off - off,\n    + \t};\n    + \tuint64_t prefix_len, suffix_len;\n    +@@ reftable/block.c: int block_iter_seek_key(struct block_iter *it, const struct block_reader *br,\n    + \t\tit->next_off = block_reader_restart_offset(br, i - 1);\n    + \telse\n    + \t\tit->next_off = br->header_off + 4;\n    +-\tit->block = br->block.data;\n    ++\tit->block = br->block_data.data;\n    + \tit->block_len = br->restart_off;\n    + \tit->hash_size = br->hash_size;\n    + \n     \n      ## reftable/block.h ##\n     @@ reftable/block.h: struct block_reader {\n    @@ reftable/block.h: struct block_reader {\n      \n      \t/* the memory block */\n     -\tstruct reftable_block block;\n    -+\tstruct reftable_block_data block;\n    ++\tstruct reftable_block_data block_data;\n      \tuint32_t hash_size;\n      \n      \t/* Uncompressed data for log entries. */\n    @@ reftable/iter.c: static void indexed_table_ref_iter_close(void *p)\n      \tstruct indexed_table_ref_iter *it = p;\n      \tblock_iter_close(&it->cur);\n     -\tblock_source_return_block(&it->block_reader.block);\n    -+\tblock_source_release_data(&it->block_reader.block);\n    ++\tblock_source_release_data(&it->block_reader.block_data);\n      \treftable_free(it->offsets);\n      \treftable_buf_release(&it->oid);\n      }\n    @@ reftable/iter.c: static int indexed_table_ref_iter_next_block(struct indexed_tab\n      \t}\n      \n     -\tblock_source_return_block(&it->block_reader.block);\n    -+\tblock_source_release_data(&it->block_reader.block);\n    ++\tblock_source_release_data(&it->block_reader.block_data);\n      \n      \toff = it->offsets[it->offset_idx++];\n      \terr = table_init_block_reader(it->table, &it->block_reader, off,\n    @@ reftable/reftable-blocksource.h: struct reftable_block {\n      \n     \n      ## reftable/table.c ##\n    +@@ reftable/table.c: static int table_iter_seek_linear(struct table_iter *ti,\n    + \t\t * as we have more than three blocks we would have an index, so\n    + \t\t * we would not do a linear search there anymore.\n    + \t\t */\n    +-\t\tmemset(&next.br.block, 0, sizeof(next.br.block));\n    ++\t\tmemset(&next.br.block_data, 0, sizeof(next.br.block_data));\n    + \t\tnext.br.zstream = NULL;\n    + \t\tnext.br.uncompressed_data = NULL;\n    + \t\tnext.br.uncompressed_cap = 0;\n     @@ reftable/table.c: int reftable_table_init_log_iterator(struct reftable_table *t,\n      int reftable_table_new(struct reftable_table **out,\n      \t\t       struct reftable_block_source *source, char const *name)\n 7:  994ac7ca02d !  7:  e6ba597eaa3 reftable/block: rename `block_reader` to `reftable_block`\n    @@ reftable/block.c: int block_reader_init(struct block_reader *br,\n      \tuint8_t block_type;\n      \tint err;\n      \n    --\terr = read_block(source, &br->block, offset, guess_block_size);\n    -+\terr = read_block(source, &block->block, offset, guess_block_size);\n    +-\terr = read_block(source, &br->block_data, offset, guess_block_size);\n    ++\terr = read_block(source, &block->block_data, offset, guess_block_size);\n      \tif (err < 0)\n      \t\tgoto done;\n      \n    --\tblock_type = br->block.data[header_size];\n    -+\tblock_type = block->block.data[header_size];\n    +-\tblock_type = br->block_data.data[header_size];\n    ++\tblock_type = block->block_data.data[header_size];\n      \tif (!reftable_is_block_type(block_type)) {\n      \t\terr = REFTABLE_FORMAT_ERROR;\n      \t\tgoto done;\n      \t}\n      \n    --\tblock_size = reftable_get_be24(br->block.data + header_size + 1);\n    -+\tblock_size = reftable_get_be24(block->block.data + header_size + 1);\n    +-\tblock_size = reftable_get_be24(br->block_data.data + header_size + 1);\n    ++\tblock_size = reftable_get_be24(block->block_data.data + header_size + 1);\n      \tif (block_size > guess_block_size) {\n    --\t\terr = read_block(source, &br->block, offset, block_size);\n    -+\t\terr = read_block(source, &block->block, offset, block_size);\n    +-\t\terr = read_block(source, &br->block_data, offset, block_size);\n    ++\t\terr = read_block(source, &block->block_data, offset, block_size);\n      \t\tif (err < 0)\n      \t\t\tgoto done;\n      \t}\n    @@ reftable/block.c: int block_reader_init(struct block_reader *br,\n      \tif (block_type == BLOCK_TYPE_LOG) {\n      \t\tuint32_t block_header_skip = 4 + header_size;\n      \t\tuLong dst_len = block_size - block_header_skip;\n    --\t\tuLong src_len = br->block.len - block_header_skip;\n    -+\t\tuLong src_len = block->block.len - block_header_skip;\n    +-\t\tuLong src_len = br->block_data.len - block_header_skip;\n    ++\t\tuLong src_len = block->block_data.len - block_header_skip;\n      \n      \t\t/* Log blocks specify the *uncompressed* size in their header. */\n     -\t\tREFTABLE_ALLOC_GROW_OR_NULL(br->uncompressed_data, block_size,\n    @@ reftable/block.c: int block_reader_init(struct block_reader *br,\n      \t\t}\n      \n      \t\t/* Copy over the block header verbatim. It's not compressed. */\n    --\t\tmemcpy(br->uncompressed_data, br->block.data, block_header_skip);\n    -+\t\tmemcpy(block->uncompressed_data, block->block.data, block_header_skip);\n    +-\t\tmemcpy(br->uncompressed_data, br->block_data.data, block_header_skip);\n    ++\t\tmemcpy(block->uncompressed_data, block->block_data.data, block_header_skip);\n      \n     -\t\tif (!br->zstream) {\n     -\t\t\tREFTABLE_CALLOC_ARRAY(br->zstream, 1);\n    @@ reftable/block.c: int block_reader_init(struct block_reader *br,\n      \t\t\tgoto done;\n      \t\t}\n      \n    --\t\tbr->zstream->next_in = br->block.data + block_header_skip;\n    +-\t\tbr->zstream->next_in = br->block_data.data + block_header_skip;\n     -\t\tbr->zstream->avail_in = src_len;\n     -\t\tbr->zstream->next_out = br->uncompressed_data + block_header_skip;\n     -\t\tbr->zstream->avail_out = dst_len;\n    -+\t\tblock->zstream->next_in = block->block.data + block_header_skip;\n    ++\t\tblock->zstream->next_in = block->block_data.data + block_header_skip;\n     +\t\tblock->zstream->avail_in = src_len;\n     +\t\tblock->zstream->next_out = block->uncompressed_data + block_header_skip;\n     +\t\tblock->zstream->avail_out = dst_len;\n    @@ reftable/block.c: int block_reader_init(struct block_reader *br,\n      \t\t}\n      \n      \t\t/* We're done with the input data. */\n    --\t\tblock_source_release_data(&br->block);\n    --\t\tbr->block.data = br->uncompressed_data;\n    --\t\tbr->block.len = block_size;\n    +-\t\tblock_source_release_data(&br->block_data);\n    +-\t\tbr->block_data.data = br->uncompressed_data;\n    +-\t\tbr->block_data.len = block_size;\n     -\t\tfull_block_size = src_len + block_header_skip - br->zstream->avail_in;\n    -+\t\tblock_source_release_data(&block->block);\n    -+\t\tblock->block.data = block->uncompressed_data;\n    -+\t\tblock->block.len = block_size;\n    ++\t\tblock_source_release_data(&block->block_data);\n    ++\t\tblock->block_data.data = block->uncompressed_data;\n    ++\t\tblock->block_data.len = block_size;\n     +\t\tfull_block_size = src_len + block_header_skip - block->zstream->avail_in;\n      \t} else if (full_block_size == 0) {\n      \t\tfull_block_size = block_size;\n    --\t} else if (block_size < full_block_size && block_size < br->block.len &&\n    --\t\t   br->block.data[block_size] != 0) {\n    -+\t} else if (block_size < full_block_size && block_size < block->block.len &&\n    -+\t\t   block->block.data[block_size] != 0) {\n    +-\t} else if (block_size < full_block_size && block_size < br->block_data.len &&\n    +-\t\t   br->block_data.data[block_size] != 0) {\n    ++\t} else if (block_size < full_block_size && block_size < block->block_data.len &&\n    ++\t\t   block->block_data.data[block_size] != 0) {\n      \t\t/* If the block is smaller than the full block size, it is\n      \t\t   padded (data followed by '\\0') or the next block is\n      \t\t   unaligned. */\n      \t\tfull_block_size = block_size;\n      \t}\n      \n    --\trestart_count = reftable_get_be16(br->block.data + block_size - 2);\n    -+\trestart_count = reftable_get_be16(block->block.data + block_size - 2);\n    +-\trestart_count = reftable_get_be16(br->block_data.data + block_size - 2);\n    ++\trestart_count = reftable_get_be16(block->block_data.data + block_size - 2);\n      \trestart_off = block_size - 2 - 3 * restart_count;\n      \n     -\tbr->block_type = block_type;\n    @@ reftable/block.c: int block_reader_init(struct block_reader *br,\n     -\tinflateEnd(br->zstream);\n     -\treftable_free(br->zstream);\n     -\treftable_free(br->uncompressed_data);\n    --\tblock_source_release_data(&br->block);\n    +-\tblock_source_release_data(&br->block_data);\n     -\tmemset(br, 0, sizeof(*br));\n     +\tinflateEnd(block->zstream);\n     +\treftable_free(block->zstream);\n     +\treftable_free(block->uncompressed_data);\n    -+\tblock_source_release_data(&block->block);\n    ++\tblock_source_release_data(&block->block_data);\n     +\tmemset(block, 0, sizeof(*block));\n      }\n      \n     -uint8_t block_reader_type(const struct block_reader *r)\n     +uint8_t reftable_block_type(const struct reftable_block *b)\n      {\n    --\treturn r->block.data[r->header_off];\n    -+\treturn b->block.data[b->header_off];\n    +-\treturn r->block_data.data[r->header_off];\n    ++\treturn b->block_data.data[b->header_off];\n      }\n      \n     -int block_reader_first_key(const struct block_reader *br, struct reftable_buf *key)\n    @@ reftable/block.c: int block_reader_init(struct block_reader *br,\n     -\tint off = br->header_off + 4, n;\n     +\tint off = block->header_off + 4, n;\n      \tstruct string_view in = {\n    --\t\t.buf = br->block.data + off,\n    +-\t\t.buf = br->block_data.data + off,\n     -\t\t.len = br->restart_off - off,\n    -+\t\t.buf = block->block.data + off,\n    ++\t\t.buf = block->block_data.data + off,\n     +\t\t.len = block->restart_off - off,\n      \t};\n      \tuint8_t extra = 0;\n    @@ reftable/block.c: int block_reader_first_key(const struct block_reader *br, stru\n     -static uint32_t block_reader_restart_offset(const struct block_reader *br, size_t idx)\n     +static uint32_t block_restart_offset(const struct reftable_block *b, size_t idx)\n      {\n    --\treturn reftable_get_be24(br->block.data + br->restart_off + 3 * idx);\n    -+\treturn reftable_get_be24(b->block.data + b->restart_off + 3 * idx);\n    +-\treturn reftable_get_be24(br->block_data.data + br->restart_off + 3 * idx);\n    ++\treturn reftable_get_be24(b->block_data.data + b->restart_off + 3 * idx);\n      }\n      \n     -void block_iter_seek_start(struct block_iter *it, const struct block_reader *br)\n     +void block_iter_seek_start(struct block_iter *it, const struct reftable_block *b)\n      {\n    --\tit->block = br->block.data;\n    +-\tit->block = br->block_data.data;\n     -\tit->block_len = br->restart_off;\n     -\tit->hash_size = br->hash_size;\n    -+\tit->block = b->block.data;\n    ++\tit->block = b->block_data.data;\n     +\tit->block_len = b->restart_off;\n     +\tit->hash_size = b->hash_size;\n      \treftable_buf_reset(&it->last_key);\n    @@ reftable/block.c: int block_reader_first_key(const struct block_reader *br, stru\n     -\tuint32_t off = block_reader_restart_offset(args->reader, idx);\n     +\tuint32_t off = block_restart_offset(args->block, idx);\n      \tstruct string_view in = {\n    --\t\t.buf = args->reader->block.data + off,\n    +-\t\t.buf = args->reader->block_data.data + off,\n     -\t\t.len = args->reader->restart_off - off,\n    -+\t\t.buf = args->block->block.data + off,\n    ++\t\t.buf = args->block->block_data.data + off,\n     +\t\t.len = args->block->restart_off - off,\n      \t};\n      \tuint64_t prefix_len, suffix_len;\n    @@ reftable/block.c: int block_iter_seek_key(struct block_iter *it, const struct bl\n     +\t\tit->next_off = block_restart_offset(block, i - 1);\n      \telse\n     -\t\tit->next_off = br->header_off + 4;\n    --\tit->block = br->block.data;\n    +-\tit->block = br->block_data.data;\n     -\tit->block_len = br->restart_off;\n     -\tit->hash_size = br->hash_size;\n     +\t\tit->next_off = block->header_off + 4;\n    -+\tit->block = block->block.data;\n    ++\tit->block = block->block_data.data;\n     +\tit->block_len = block->restart_off;\n     +\tit->hash_size = block->hash_size;\n      \n    @@ reftable/iter.c: static void indexed_table_ref_iter_close(void *p)\n      {\n      \tstruct indexed_table_ref_iter *it = p;\n      \tblock_iter_close(&it->cur);\n    --\tblock_source_release_data(&it->block_reader.block);\n    -+\tblock_source_release_data(&it->block.block);\n    +-\tblock_source_release_data(&it->block_reader.block_data);\n    ++\tblock_source_release_data(&it->block.block_data);\n      \treftable_free(it->offsets);\n      \treftable_buf_release(&it->oid);\n      }\n    @@ reftable/iter.c: static int indexed_table_ref_iter_next_block(struct indexed_tab\n      \t\treturn 1;\n      \t}\n      \n    --\tblock_source_release_data(&it->block_reader.block);\n    -+\tblock_source_release_data(&it->block.block);\n    +-\tblock_source_release_data(&it->block_reader.block_data);\n    ++\tblock_source_release_data(&it->block.block_data);\n      \n      \toff = it->offsets[it->offset_idx++];\n     -\terr = table_init_block_reader(it->table, &it->block_reader, off,\n    @@ reftable/table.c: static int table_iter_seek_linear(struct table_iter *ti,\n      \t\t * as we have more than three blocks we would have an index, so\n      \t\t * we would not do a linear search there anymore.\n      \t\t */\n    --\t\tmemset(&next.br.block, 0, sizeof(next.br.block));\n    +-\t\tmemset(&next.br.block_data, 0, sizeof(next.br.block_data));\n     -\t\tnext.br.zstream = NULL;\n     -\t\tnext.br.uncompressed_data = NULL;\n     -\t\tnext.br.uncompressed_cap = 0;\n    -+\t\tmemset(&next.block.block, 0, sizeof(next.block.block));\n    ++\t\tmemset(&next.block.block_data, 0, sizeof(next.block.block_data));\n     +\t\tnext.block.zstream = NULL;\n     +\t\tnext.block.uncompressed_data = NULL;\n     +\t\tnext.block.uncompressed_cap = 0;\n 8:  17110157b97 =  8:  9900fd1550a git-zlib: use `struct z_stream_s` instead of typedef\n 9:  a93b32ed3ca !  9:  8f61a428774 reftable/block: create public interface for reading blocks\n    @@ reftable/block.h: int block_writer_finish(struct block_writer *w);\n     -\tuint32_t header_off;\n     -\n     -\t/* the memory block */\n    --\tstruct reftable_block_data block;\n    +-\tstruct reftable_block_data block_data;\n     -\tuint32_t hash_size;\n     -\n     -\t/* Uncompressed data for log entries. */\n    @@ reftable/reftable-block.h (new)\n     +\tuint32_t header_off;\n     +\n     +\t/* The memory block. */\n    -+\tstruct reftable_block_data block;\n    ++\tstruct reftable_block_data block_data;\n     +\tuint32_t hash_size;\n     +\n     +\t/* Uncompressed data for log entries. */\n10:  82c507d95c6 ! 10:  fb78ed25b84 reftable/block: store block pointer in the block iterator\n    @@ Commit message\n             confusion.\n     \n           - The lifetime of the block iterator is tied to the block anyway, but\n    -        we hide that a bit by only storing pointers into the block.\n    +        we hide that a bit by only storing pointers pointing into the block.\n     \n         There isn't really any good reason why we rip out parts of the block\n         instead of storing a pointer to the block itself.\n    @@ Commit message\n     \n      ## reftable/block.c ##\n     @@ reftable/block.c: static uint32_t block_restart_offset(const struct reftable_block *b, size_t idx)\n    - \treturn reftable_get_be24(b->block.data + b->restart_off + 3 * idx);\n    + \treturn reftable_get_be24(b->block_data.data + b->restart_off + 3 * idx);\n      }\n      \n     -void block_iter_seek_start(struct block_iter *it, const struct reftable_block *b)\n     +void block_iter_seek_start(struct block_iter *it, const struct reftable_block *block)\n      {\n    --\tit->block = b->block.data;\n    +-\tit->block = b->block_data.data;\n     -\tit->block_len = b->restart_off;\n     -\tit->hash_size = b->hash_size;\n     +\tit->block = block;\n    @@ reftable/block.c: static int restart_needle_less(size_t idx, void *_args)\n      \tstruct string_view in = {\n     -\t\t.buf = (unsigned char *) it->block + it->next_off,\n     -\t\t.len = it->block_len - it->next_off,\n    -+\t\t.buf = (unsigned char *) it->block->block.data + it->next_off,\n    ++\t\t.buf = (unsigned char *) it->block->block_data.data + it->next_off,\n     +\t\t.len = it->block->restart_off - it->next_off,\n      \t};\n      \tstruct string_view start = in;\n    @@ reftable/block.c: int block_iter_seek_key(struct block_iter *it, const struct re\n      \t\tit->next_off = block_restart_offset(block, i - 1);\n      \telse\n      \t\tit->next_off = block->header_off + 4;\n    --\tit->block = block->block.data;\n    +-\tit->block = block->block_data.data;\n     -\tit->block_len = block->restart_off;\n     -\tit->hash_size = block->hash_size;\n     +\tit->block = block;\n11:  132c09cff1f ! 11:  cc990effb4f reftable/block: make block iterators reseekable\n    @@ Commit message\n         Refactor the block iterators so that initialization and seeking are\n         different from one another. This makes the iterator trivially reseekable\n         by storing the pointer to the block at initialization time, which we can\n    -    then reuse on ever seek.\n    +    then reuse on every seek.\n     \n         This refactoring prepares the code for exposing a `reftable_iterator`\n         interface for blocks in a subsequent commit. Callsites are adjusted\n    @@ Commit message\n     \n      ## reftable/block.c ##\n     @@ reftable/block.c: static uint32_t block_restart_offset(const struct reftable_block *b, size_t idx)\n    - \treturn reftable_get_be24(b->block.data + b->restart_off + 3 * idx);\n    + \treturn reftable_get_be24(b->block_data.data + b->restart_off + 3 * idx);\n      }\n      \n     -void block_iter_seek_start(struct block_iter *it, const struct reftable_block *block)\n    @@ t/unit-tests/t-reftable-block.c: static void t_ref_block_read_write(void)\n      \n     -\tblock_iter_seek_start(&it, &block);\n     +\tblock_iter_init(&it, &block);\n    -+\tblock_iter_seek_start(&it);\n      \n      \tfor (i = 0; ; i++) {\n      \t\tret = block_iter_next(&it, &rec);\n    @@ t/unit-tests/t-reftable-block.c: static void t_log_block_read_write(void)\n      \n     -\tblock_iter_seek_start(&it, &block);\n     +\tblock_iter_init(&it, &block);\n    -+\tblock_iter_seek_start(&it);\n      \n      \tfor (i = 0; ; i++) {\n      \t\tret = block_iter_next(&it, &rec);\n    @@ t/unit-tests/t-reftable-block.c: static void t_obj_block_read_write(void)\n      \n     -\tblock_iter_seek_start(&it, &block);\n     +\tblock_iter_init(&it, &block);\n    -+\tblock_iter_seek_start(&it);\n      \n      \tfor (i = 0; ; i++) {\n      \t\tret = block_iter_next(&it, &rec);\n    @@ t/unit-tests/t-reftable-block.c: static void t_index_block_read_write(void)\n      \n     -\tblock_iter_seek_start(&it, &block);\n     +\tblock_iter_init(&it, &block);\n    -+\tblock_iter_seek_start(&it);\n      \n      \tfor (i = 0; ; i++) {\n      \t\tret = block_iter_next(&it, &rec);\n12:  cc4fd9e7a16 = 12:  43dda28cf3a reftable/block: expose a generic iterator over reftable records\n13:  e022bc4cb24 = 13:  d80412e8806 reftable/table: add `reftable_table` to the public interface\n14:  0adde9c6f4a ! 14:  f12ded11167 reftable/table: introduce iterator for table blocks\n    @@ reftable/reftable-table.h: uint64_t reftable_table_min_update_index(struct refta\n     +\tvoid *iter_arg;\n     +};\n     +\n    -+int reftable_table_init_table_iterator(struct reftable_table *t,\n    -+\t\t\t\t       struct reftable_table_iterator *it);\n    ++int reftable_table_iterator_init(struct reftable_table_iterator *it,\n    ++\t\t\t\t struct reftable_table *t);\n     +\n     +void reftable_table_iterator_release(struct reftable_table_iterator *it);\n     +\n    @@ reftable/table.c: int reftable_table_print_blocks(const char *tablename)\n      \treturn err;\n      }\n     +\n    -+int reftable_table_init_table_iterator(struct reftable_table *t,\n    -+\t\t\t\t       struct reftable_table_iterator *it)\n    ++int reftable_table_iterator_init(struct reftable_table_iterator *it,\n    ++\t\t\t\t struct reftable_table *t)\n     +{\n     +\tstruct table_iter *ti;\n     +\tint err;\n    @@ t/unit-tests/t-reftable-table.c: static int t_table_reseek(void)\n     +\n     +\tREFTABLE_CALLOC_ARRAY(records, nrecords);\n     +\tfor (size_t i = 0; i < nrecords; i++) {\n    -+\t\trecords[i].value_type = REFTABLE_REF_VAL1,\n    -+\t\trecords[i].refname = xstrfmt(\"refs/heads/branch-%03\"PRIuMAX, (uintmax_t) i);\n    ++\t\trecords[i].value_type = REFTABLE_REF_VAL1;\n    ++\t\trecords[i].refname = xstrfmt(\"refs/heads/branch-%03\"PRIuMAX,\n    ++\t\t\t\t\t     (uintmax_t) i);\n     +\t}\n     +\n     +\tt_reftable_write_to_buf(&buf, records, nrecords, NULL, 0, NULL);\n    @@ t/unit-tests/t-reftable-table.c: static int t_table_reseek(void)\n     +\tret = reftable_table_new(&table, &source, \"name\");\n     +\tcheck(!ret);\n     +\n    -+\tret = reftable_table_init_table_iterator(table, &it);\n    ++\tret = reftable_table_iterator_init(&it, table);\n     +\tcheck(!ret);\n     +\n     +\tfor (size_t i = 0; i < ARRAY_SIZE(expected_blocks); i++) {\n15:  71be6c4a8f7 ! 15:  abc886e821d reftable/constants: make block types part of the public interface\n    @@ reftable/block.c: int reftable_block_init(struct reftable_block *block,\n     +\tif (block_type == REFTABLE_BLOCK_TYPE_LOG) {\n      \t\tuint32_t block_header_skip = 4 + header_size;\n      \t\tuLong dst_len = block_size - block_header_skip;\n    - \t\tuLong src_len = block->block.len - block_header_skip;\n    + \t\tuLong src_len = block->block_data.len - block_header_skip;\n     \n      ## reftable/constants.h ##\n     @@\n    @@ reftable/constants.h\n     \n      ## reftable/iter.c ##\n     @@ reftable/iter.c: static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n    - \tblock_source_release_data(&it->block.block);\n    + \tblock_source_release_data(&it->block.block_data);\n      \n      \toff = it->offsets[it->offset_idx++];\n     -\terr = table_init_block(it->table, &it->block, off, BLOCK_TYPE_REF);\n16:  b4e1e25ef86 ! 16:  6a0d891ae76 reftable/table: move printing logic into test helper\n    @@ reftable/table.c: uint64_t reftable_table_min_update_index(struct reftable_table\n     -\treturn err;\n     -}\n     -\n    - int reftable_table_init_table_iterator(struct reftable_table *t,\n    - \t\t\t\t       struct reftable_table_iterator *it)\n    + int reftable_table_iterator_init(struct reftable_table_iterator *it,\n    + \t\t\t\t struct reftable_table *t)\n      {\n     \n      ## t/helper/test-reftable.c ##\n    @@ t/helper/test-reftable.c: static void print_help(void)\n     +\tif (err < 0)\n     +\t\tgoto done;\n     +\n    -+\terr = reftable_table_init_table_iterator(table, &ti);\n    ++\terr = reftable_table_iterator_init(&ti, table);\n     +\tif (err < 0)\n     +\t\tgoto done;\n     +\n\n---\nbase-commit: bc705c20b9a88abaff0f379bab6d545f012656af\nchange-id: 20241210-pks-reftable-polishing-332ce318cdea\n\n"},{"id":"515764","messageId":"20250407-pks-reftable-polishing-v2-1-316c4ff10571@pks.im","threadId":"63222","inReplyTo":"20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im","subject":"[PATCH v2 01/16] reftable: fix formatting of the license header","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T13:16:13Z","receivedAt":"2025-04-07T13:16:25Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The license headers used across the reftable library doesn't follow our\ntypical coding style for multi-line comments. Fix it.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/basics.c               | 12 ++++++------\n reftable/basics.h               | 12 ++++++------\n reftable/block.c                | 12 ++++++------\n reftable/block.h                | 12 ++++++------\n reftable/blocksource.c          | 12 ++++++------\n reftable/blocksource.h          | 12 ++++++------\n reftable/constants.h            | 12 ++++++------\n reftable/error.c                | 12 ++++++------\n reftable/iter.c                 | 12 ++++++------\n reftable/iter.h                 | 12 ++++++------\n reftable/merged.c               | 12 ++++++------\n reftable/merged.h               | 12 ++++++------\n reftable/pq.c                   | 12 ++++++------\n reftable/pq.h                   | 12 ++++++------\n reftable/reader.c               | 12 ++++++------\n reftable/reader.h               | 12 ++++++------\n reftable/record.c               | 12 ++++++------\n reftable/record.h               | 12 ++++++------\n reftable/reftable-basics.h      |  2 +-\n reftable/reftable-blocksource.h | 12 ++++++------\n reftable/reftable-error.h       | 12 ++++++------\n reftable/reftable-iterator.h    | 12 ++++++------\n reftable/reftable-merged.h      | 12 ++++++------\n reftable/reftable-reader.h      | 12 ++++++------\n reftable/reftable-record.h      | 12 ++++++------\n reftable/reftable-stack.h       | 12 ++++++------\n reftable/reftable-writer.h      | 12 ++++++------\n reftable/stack.c                | 12 ++++++------\n reftable/stack.h                | 12 ++++++------\n reftable/system.h               | 12 ++++++------\n reftable/tree.c                 | 12 ++++++------\n reftable/tree.h                 | 12 ++++++------\n reftable/writer.c               | 12 ++++++------\n reftable/writer.h               | 12 ++++++------\n 34 files changed, 199 insertions(+), 199 deletions(-)\n\ndiff --git a/reftable/basics.c b/reftable/basics.c\nindex 8c4a4433e41..9988ebd635e 100644\n--- a/reftable/basics.c\n+++ b/reftable/basics.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #define REFTABLE_ALLOW_BANNED_ALLOCATORS\n #include \"basics.h\"\ndiff --git a/reftable/basics.h b/reftable/basics.h\nindex fd59cbb7720..96a2f0d3829 100644\n--- a/reftable/basics.h\n+++ b/reftable/basics.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef BASICS_H\n #define BASICS_H\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 53b5e044690..a5734d44415 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"block.h\"\n \ndiff --git a/reftable/block.h b/reftable/block.h\nindex bef2b8a4c5c..eaeffdffc90 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef BLOCK_H\n #define BLOCK_H\ndiff --git a/reftable/blocksource.c b/reftable/blocksource.c\nindex 78c1be23373..1397cbe7800 100644\n--- a/reftable/blocksource.c\n+++ b/reftable/blocksource.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"system.h\"\n \ndiff --git a/reftable/blocksource.h b/reftable/blocksource.h\nindex a84a3ccd891..7b67898ae22 100644\n--- a/reftable/blocksource.h\n+++ b/reftable/blocksource.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef BLOCKSOURCE_H\n #define BLOCKSOURCE_H\ndiff --git a/reftable/constants.h b/reftable/constants.h\nindex f6beb843ebf..091728cf033 100644\n--- a/reftable/constants.h\n+++ b/reftable/constants.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef CONSTANTS_H\n #define CONSTANTS_H\ndiff --git a/reftable/error.c b/reftable/error.c\nindex 660d0296170..c7cab2dbc42 100644\n--- a/reftable/error.c\n+++ b/reftable/error.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"system.h\"\n #include \"reftable-error.h\"\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex f520382e700..44a155e756e 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"iter.h\"\n \ndiff --git a/reftable/iter.h b/reftable/iter.h\nindex 40f98893b85..bd217405f55 100644\n--- a/reftable/iter.h\n+++ b/reftable/iter.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef ITER_H\n #define ITER_H\ndiff --git a/reftable/merged.c b/reftable/merged.c\nindex 4ff1553772a..1829a081756 100644\n--- a/reftable/merged.c\n+++ b/reftable/merged.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"merged.h\"\n \ndiff --git a/reftable/merged.h b/reftable/merged.h\nindex 0b7d939e92b..ae92409d342 100644\n--- a/reftable/merged.h\n+++ b/reftable/merged.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef MERGED_H\n #define MERGED_H\ndiff --git a/reftable/pq.c b/reftable/pq.c\nindex 82394a972db..9a79f5c5eec 100644\n--- a/reftable/pq.c\n+++ b/reftable/pq.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"pq.h\"\n \ndiff --git a/reftable/pq.h b/reftable/pq.h\nindex ff39016445b..42310670b09 100644\n--- a/reftable/pq.h\n+++ b/reftable/pq.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef PQ_H\n #define PQ_H\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 172aff2c10b..155a26c30ff 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"reader.h\"\n \ndiff --git a/reftable/reader.h b/reftable/reader.h\nindex bb72108a6f1..c9dccbd6c97 100644\n--- a/reftable/reader.h\n+++ b/reftable/reader.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef READER_H\n #define READER_H\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 142853d5070..26cd834d405 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n /* record.c - methods for different types of records. */\n \ndiff --git a/reftable/record.h b/reftable/record.h\nindex 867810a9328..7953f352a3c 100644\n--- a/reftable/record.h\n+++ b/reftable/record.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef RECORD_H\n #define RECORD_H\ndiff --git a/reftable/reftable-basics.h b/reftable/reftable-basics.h\nindex e0397ed5836..ed7c7c9ac2c 100644\n--- a/reftable/reftable-basics.h\n+++ b/reftable/reftable-basics.h\n@@ -4,7 +4,7 @@\n  * Use of this source code is governed by a BSD-style\n  * license that can be found in the LICENSE file or at\n  * https://developers.google.com/open-source/licenses/bsd\n-*/\n+ */\n \n #ifndef REFTABLE_BASICS_H\n #define REFTABLE_BASICS_H\ndiff --git a/reftable/reftable-blocksource.h b/reftable/reftable-blocksource.h\nindex 6b326aa5ea5..8692cd017e9 100644\n--- a/reftable/reftable-blocksource.h\n+++ b/reftable/reftable-blocksource.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef REFTABLE_BLOCKSOURCE_H\n #define REFTABLE_BLOCKSOURCE_H\ndiff --git a/reftable/reftable-error.h b/reftable/reftable-error.h\nindex a7e33d964d0..d100e0df927 100644\n--- a/reftable/reftable-error.h\n+++ b/reftable/reftable-error.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef REFTABLE_ERROR_H\n #define REFTABLE_ERROR_H\ndiff --git a/reftable/reftable-iterator.h b/reftable/reftable-iterator.h\nindex e3bf688d53d..af582028c27 100644\n--- a/reftable/reftable-iterator.h\n+++ b/reftable/reftable-iterator.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef REFTABLE_ITERATOR_H\n #define REFTABLE_ITERATOR_H\ndiff --git a/reftable/reftable-merged.h b/reftable/reftable-merged.h\nindex f2d01c3ef82..445756e475a 100644\n--- a/reftable/reftable-merged.h\n+++ b/reftable/reftable-merged.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef REFTABLE_MERGED_H\n #define REFTABLE_MERGED_H\ndiff --git a/reftable/reftable-reader.h b/reftable/reftable-reader.h\nindex 0085fbb9032..9b6a8ac0716 100644\n--- a/reftable/reftable-reader.h\n+++ b/reftable/reftable-reader.h\n@@ -1,10 +1,10 @@\n /*\n-  Copyright 2020 Google LLC\n-\n-  Use of this source code is governed by a BSD-style\n-  license that can be found in the LICENSE file or at\n-  https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef REFTABLE_READER_H\n #define REFTABLE_READER_H\ndiff --git a/reftable/reftable-record.h b/reftable/reftable-record.h\nindex 931e5947441..385a74cc864 100644\n--- a/reftable/reftable-record.h\n+++ b/reftable/reftable-record.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef REFTABLE_RECORD_H\n #define REFTABLE_RECORD_H\ndiff --git a/reftable/reftable-stack.h b/reftable/reftable-stack.h\nindex ae14270ea74..910ec6ef3a2 100644\n--- a/reftable/reftable-stack.h\n+++ b/reftable/reftable-stack.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef REFTABLE_STACK_H\n #define REFTABLE_STACK_H\ndiff --git a/reftable/reftable-writer.h b/reftable/reftable-writer.h\nindex 1befe3b07cf..0fbeff17f46 100644\n--- a/reftable/reftable-writer.h\n+++ b/reftable/reftable-writer.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef REFTABLE_WRITER_H\n #define REFTABLE_WRITER_H\ndiff --git a/reftable/stack.c b/reftable/stack.c\nindex 6dac015b473..bc3cfa91170 100644\n--- a/reftable/stack.c\n+++ b/reftable/stack.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"stack.h\"\n \ndiff --git a/reftable/stack.h b/reftable/stack.h\nindex 5b45cff4f75..d91f5ce6072 100644\n--- a/reftable/stack.h\n+++ b/reftable/stack.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef STACK_H\n #define STACK_H\ndiff --git a/reftable/system.h b/reftable/system.h\nindex 10055fbff2d..cf207aa0c53 100644\n--- a/reftable/system.h\n+++ b/reftable/system.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef SYSTEM_H\n #define SYSTEM_H\ndiff --git a/reftable/tree.c b/reftable/tree.c\nindex f4dbe720901..a52f7c0c7d4 100644\n--- a/reftable/tree.c\n+++ b/reftable/tree.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"system.h\"\n #include \"tree.h\"\ndiff --git a/reftable/tree.h b/reftable/tree.h\nindex 9604453b6d5..2c9c4652994 100644\n--- a/reftable/tree.h\n+++ b/reftable/tree.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef TREE_H\n #define TREE_H\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex ce55a1deb06..f0accfd0c32 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #include \"writer.h\"\n \ndiff --git a/reftable/writer.h b/reftable/writer.h\nindex 1f4788a430c..9f53610b27a 100644\n--- a/reftable/writer.h\n+++ b/reftable/writer.h\n@@ -1,10 +1,10 @@\n /*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n \n #ifndef WRITER_H\n #define WRITER_H\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515767","messageId":"20250407-pks-reftable-polishing-v2-3-316c4ff10571@pks.im","threadId":"63222","inReplyTo":"20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im","subject":"[PATCH v2 03/16] reftable/blocksource: consolidate code into a single file","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T13:16:15Z","receivedAt":"2025-04-07T13:16:26Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The code that implements block sources is distributed across a couple of\nfiles. Consolidate all of it into \"reftable/blocksource.c\" and its\naccompanying header so that it is easier to locate and more self\ncontained.\n\nWhile at it, rename some of the functions to have properly scoped names.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/block.c                    | 17 +++--------------\n reftable/block.h                    |  3 ---\n reftable/blocksource.c              | 35 +++++++++++++++++++++++++++++++++++\n reftable/blocksource.h              | 27 ++++++++++++++++++++++++++-\n reftable/iter.c                     |  5 +++--\n reftable/reftable-blocksource.h     |  3 ++-\n reftable/table.c                    | 33 +++++----------------------------\n reftable/table.h                    |  7 -------\n t/unit-tests/t-reftable-block.c     |  8 ++++----\n t/unit-tests/t-reftable-readwrite.c |  4 ++--\n 10 files changed, 80 insertions(+), 62 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex a5734d44415..97740187259 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -221,7 +221,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tuint32_t restart_start = 0;\n \tuint8_t *restart_bytes = NULL;\n \n-\treftable_block_done(&br->block);\n+\tblock_source_return_block(&br->block);\n \n \tif (!reftable_is_block_type(typ)) {\n \t\terr =  REFTABLE_FORMAT_ERROR;\n@@ -285,7 +285,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\t}\n \n \t\t/* We're done with the input data. */\n-\t\treftable_block_done(block);\n+\t\tblock_source_return_block(block);\n \t\tblock->data = br->uncompressed_data;\n \t\tblock->len = sz;\n \t\tfull_block_size = src_len + block_header_skip - br->zstream->avail_in;\n@@ -324,7 +324,7 @@ void block_reader_release(struct block_reader *br)\n \tinflateEnd(br->zstream);\n \treftable_free(br->zstream);\n \treftable_free(br->uncompressed_data);\n-\treftable_block_done(&br->block);\n+\tblock_source_return_block(&br->block);\n }\n \n uint8_t block_reader_type(const struct block_reader *r)\n@@ -570,14 +570,3 @@ void block_writer_release(struct block_writer *bw)\n \treftable_buf_release(&bw->last_key);\n \t/* the block is not owned. */\n }\n-\n-void reftable_block_done(struct reftable_block *blockp)\n-{\n-\tstruct reftable_block_source source = blockp->source;\n-\tif (blockp && source.ops)\n-\t\tsource.ops->return_block(source.arg, blockp);\n-\tblockp->data = NULL;\n-\tblockp->len = 0;\n-\tblockp->source.ops = NULL;\n-\tblockp->source.arg = NULL;\n-}\ndiff --git a/reftable/block.h b/reftable/block.h\nindex eaeffdffc90..203b07d9a44 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -142,7 +142,4 @@ size_t header_size(int version);\n /* size of file footer, depending on format version */\n size_t footer_size(int version);\n \n-/* returns a block to its source. */\n-void reftable_block_done(struct reftable_block *ret);\n-\n #endif\ndiff --git a/reftable/blocksource.c b/reftable/blocksource.c\nindex 1397cbe7800..bc785506fb1 100644\n--- a/reftable/blocksource.c\n+++ b/reftable/blocksource.c\n@@ -13,6 +13,41 @@\n #include \"reftable-blocksource.h\"\n #include \"reftable-error.h\"\n \n+void block_source_return_block(struct reftable_block *block)\n+{\n+\tstruct reftable_block_source source = block->source;\n+\tif (block && source.ops)\n+\t\tsource.ops->return_block(source.arg, block);\n+\tblock->data = NULL;\n+\tblock->len = 0;\n+\tblock->source.ops = NULL;\n+\tblock->source.arg = NULL;\n+}\n+\n+void block_source_close(struct reftable_block_source *source)\n+{\n+\tif (!source->ops) {\n+\t\treturn;\n+\t}\n+\n+\tsource->ops->close(source->arg);\n+\tsource->ops = NULL;\n+}\n+\n+ssize_t block_source_read_block(struct reftable_block_source *source,\n+\t\t\t\tstruct reftable_block *dest, uint64_t off,\n+\t\t\t\tuint32_t size)\n+{\n+\tssize_t result = source->ops->read_block(source->arg, dest, off, size);\n+\tdest->source = *source;\n+\treturn result;\n+}\n+\n+uint64_t block_source_size(struct reftable_block_source *source)\n+{\n+\treturn source->ops->size(source->arg);\n+}\n+\n static void reftable_buf_return_block(void *b REFTABLE_UNUSED, struct reftable_block *dest)\n {\n \tif (dest->len)\ndiff --git a/reftable/blocksource.h b/reftable/blocksource.h\nindex 7b67898ae22..639b9a1a3c5 100644\n--- a/reftable/blocksource.h\n+++ b/reftable/blocksource.h\n@@ -12,9 +12,34 @@\n #include \"system.h\"\n \n struct reftable_block_source;\n+struct reftable_block;\n struct reftable_buf;\n \n-/* Create an in-memory block source for reading reftables */\n+/*\n+ * Close the block source and the underlying resource. This is a no-op in case\n+ * the block source is zero-initialized.\n+ */\n+void block_source_close(struct reftable_block_source *source);\n+\n+/*\n+ * Read a block of length `size` from the source at the given `off`.\n+ */\n+ssize_t block_source_read_block(struct reftable_block_source *source,\n+\t\t\t\tstruct reftable_block *dest, uint64_t off,\n+\t\t\t\tuint32_t size);\n+\n+/*\n+ * Return the total length of the underlying resource.\n+ */\n+uint64_t block_source_size(struct reftable_block_source *source);\n+\n+/*\n+ * Return a block to its original source, releasing any resources associated\n+ * with it.\n+ */\n+void block_source_return_block(struct reftable_block *block);\n+\n+/* Create an in-memory block source for reading reftables. */\n void block_source_from_buf(struct reftable_block_source *bs,\n \t\t\t   struct reftable_buf *buf);\n \ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex 7376f263c99..6af6eb49396 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -11,6 +11,7 @@\n #include \"system.h\"\n \n #include \"block.h\"\n+#include \"blocksource.h\"\n #include \"constants.h\"\n #include \"reftable-error.h\"\n #include \"table.h\"\n@@ -113,7 +114,7 @@ static void indexed_table_ref_iter_close(void *p)\n {\n \tstruct indexed_table_ref_iter *it = p;\n \tblock_iter_close(&it->cur);\n-\treftable_block_done(&it->block_reader.block);\n+\tblock_source_return_block(&it->block_reader.block);\n \treftable_free(it->offsets);\n \treftable_buf_release(&it->oid);\n }\n@@ -127,7 +128,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n \t\treturn 1;\n \t}\n \n-\treftable_block_done(&it->block_reader.block);\n+\tblock_source_return_block(&it->block_reader.block);\n \n \toff = it->offsets[it->offset_idx++];\n \terr = table_init_block_reader(it->table, &it->block_reader, off,\ndiff --git a/reftable/reftable-blocksource.h b/reftable/reftable-blocksource.h\nindex 8692cd017e9..96430b629e4 100644\n--- a/reftable/reftable-blocksource.h\n+++ b/reftable/reftable-blocksource.h\n@@ -11,7 +11,8 @@\n \n #include <stdint.h>\n \n-/* block_source is a generic wrapper for a seekable readable file.\n+/*\n+ * Generic wrapper for a seekable readable file.\n  */\n struct reftable_block_source {\n \tstruct reftable_block_source_vtable *ops;\ndiff --git a/reftable/table.c b/reftable/table.c\nindex 440fb559ad1..d18e17b0d44 100644\n--- a/reftable/table.c\n+++ b/reftable/table.c\n@@ -10,35 +10,12 @@\n \n #include \"system.h\"\n #include \"block.h\"\n+#include \"blocksource.h\"\n #include \"constants.h\"\n #include \"iter.h\"\n #include \"record.h\"\n #include \"reftable-error.h\"\n \n-uint64_t block_source_size(struct reftable_block_source *source)\n-{\n-\treturn source->ops->size(source->arg);\n-}\n-\n-ssize_t block_source_read_block(struct reftable_block_source *source,\n-\t\t\t\tstruct reftable_block *dest, uint64_t off,\n-\t\t\t\tuint32_t size)\n-{\n-\tssize_t result = source->ops->read_block(source->arg, dest, off, size);\n-\tdest->source = *source;\n-\treturn result;\n-}\n-\n-void block_source_close(struct reftable_block_source *source)\n-{\n-\tif (!source->ops) {\n-\t\treturn;\n-\t}\n-\n-\tsource->ops->close(source->arg);\n-\tsource->ops = NULL;\n-}\n-\n static struct reftable_table_offsets *\n table_offsets_for(struct reftable_table *t, uint8_t typ)\n {\n@@ -249,7 +226,7 @@ int table_init_block_reader(struct reftable_table *t, struct block_reader *br,\n \t}\n \n \tif (block_size > guess_block_size) {\n-\t\treftable_block_done(&block);\n+\t\tblock_source_return_block(&block);\n \t\terr = table_get_block(t, &block, next_off, block_size);\n \t\tif (err < 0) {\n \t\t\tgoto done;\n@@ -259,7 +236,7 @@ int table_init_block_reader(struct reftable_table *t, struct block_reader *br,\n \terr = block_reader_init(br, &block, header_off, t->block_size,\n \t\t\t\thash_size(t->hash_id));\n done:\n-\treftable_block_done(&block);\n+\tblock_source_return_block(&block);\n \n \treturn err;\n }\n@@ -666,8 +643,8 @@ int reftable_table_new(struct reftable_table **out,\n \t*out = t;\n \n done:\n-\treftable_block_done(&footer);\n-\treftable_block_done(&header);\n+\tblock_source_return_block(&footer);\n+\tblock_source_return_block(&header);\n \tif (err) {\n \t\tif (t)\n \t\t\treftable_free(t->name);\ndiff --git a/reftable/table.h b/reftable/table.h\nindex 9cd8f80a207..8d8dd2b413d 100644\n--- a/reftable/table.h\n+++ b/reftable/table.h\n@@ -14,13 +14,6 @@\n #include \"reftable-iterator.h\"\n #include \"reftable-table.h\"\n \n-uint64_t block_source_size(struct reftable_block_source *source);\n-\n-ssize_t block_source_read_block(struct reftable_block_source *source,\n-\t\t\t\tstruct reftable_block *dest, uint64_t off,\n-\t\t\t\tuint32_t size);\n-void block_source_close(struct reftable_block_source *source);\n-\n /* metadata for a block type */\n struct reftable_table_offsets {\n \tint is_present;\ndiff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c\nindex 22040aeefa5..8bb40482347 100644\n--- a/t/unit-tests/t-reftable-block.c\n+++ b/t/unit-tests/t-reftable-block.c\n@@ -100,7 +100,7 @@ static void t_ref_block_read_write(void)\n \tblock_reader_release(&br);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n-\treftable_block_done(&br.block);\n+\tblock_source_return_block(&br.block);\n \treftable_buf_release(&want);\n \treftable_buf_release(&buf);\n \tfor (i = 0; i < N; i++)\n@@ -190,7 +190,7 @@ static void t_log_block_read_write(void)\n \tblock_reader_release(&br);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n-\treftable_block_done(&br.block);\n+\tblock_source_return_block(&br.block);\n \treftable_buf_release(&want);\n \treftable_buf_release(&buf);\n \tfor (i = 0; i < N; i++)\n@@ -273,7 +273,7 @@ static void t_obj_block_read_write(void)\n \tblock_reader_release(&br);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n-\treftable_block_done(&br.block);\n+\tblock_source_return_block(&br.block);\n \treftable_buf_release(&want);\n \treftable_buf_release(&buf);\n \tfor (i = 0; i < N; i++)\n@@ -365,7 +365,7 @@ static void t_index_block_read_write(void)\n \tblock_reader_release(&br);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n-\treftable_block_done(&br.block);\n+\tblock_source_return_block(&br.block);\n \treftable_buf_release(&want);\n \treftable_buf_release(&buf);\n \tfor (i = 0; i < N; i++)\ndiff --git a/t/unit-tests/t-reftable-readwrite.c b/t/unit-tests/t-reftable-readwrite.c\nindex c4c27242ba9..3fba888cdaa 100644\n--- a/t/unit-tests/t-reftable-readwrite.c\n+++ b/t/unit-tests/t-reftable-readwrite.c\n@@ -32,13 +32,13 @@ static void t_buffer(void)\n \tn = block_source_read_block(&source, &out, 0, sizeof(in));\n \tcheck_int(n, ==, sizeof(in));\n \tcheck(!memcmp(in, out.data, n));\n-\treftable_block_done(&out);\n+\tblock_source_return_block(&out);\n \n \tn = block_source_read_block(&source, &out, 1, 2);\n \tcheck_int(n, ==, 2);\n \tcheck(!memcmp(out.data, \"el\", 2));\n \n-\treftable_block_done(&out);\n+\tblock_source_return_block(&out);\n \tblock_source_close(&source);\n \treftable_buf_release(&buf);\n }\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515769","messageId":"20250407-pks-reftable-polishing-v2-2-316c4ff10571@pks.im","threadId":"63222","inReplyTo":"20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im","subject":"[PATCH v2 02/16] reftable/reader: rename data structure to \"table\"","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T13:16:14Z","receivedAt":"2025-04-07T13:16:26Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The `struct reftable_reader` subsystem encapsulates a table that has\nbeen read from the disk. As such, the current name of that structure is\nsomewhat hard to understand as it only talks about the fact that we read\nsomething from disk, without really giving an indicator _what_ that is.\n\nFurthermore, this naming schema doesn't really fit well into how the\nother structures are named: `reftable_merged_table`, `reftable_stack`,\n`reftable_block` and `reftable_record` are all named after what they\nencapsulate.\n\nRename the subsystem to `reftable_table`, which directly gives a hint\nthat the data structure is about handling the individual tables part of\nthe stack.\n\nWhile this change results in a lot of churn, it prepares for us exposing\nthe APIs to third-party callers now that the reftable library is a\nstandalone library that can be linked against by other projects.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n Makefile                                           |   4 +-\n meson.build                                        |   2 +-\n reftable/iter.c                                    |  10 +-\n reftable/iter.h                                    |   4 +-\n reftable/merged.c                                  |  26 +--\n reftable/merged.h                                  |   4 +-\n reftable/reftable-merged.h                         |   6 +-\n reftable/reftable-reader.h                         |  72 ------\n reftable/reftable-table.h                          |  72 ++++++\n reftable/stack.c                                   | 168 +++++++-------\n reftable/stack.h                                   |   4 +-\n reftable/{reader.c => table.c}                     | 254 ++++++++++-----------\n reftable/{reader.h => table.h}                     |  30 +--\n t/helper/test-reftable.c                           |  14 +-\n t/meson.build                                      |   2 +-\n t/unit-tests/t-reftable-merged.c                   |  74 +++---\n t/unit-tests/t-reftable-readwrite.c                |  96 ++++----\n t/unit-tests/t-reftable-stack.c                    |  66 +++---\n .../{t-reftable-reader.c => t-reftable-table.c}    |  26 +--\n 19 files changed, 467 insertions(+), 467 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 22d632da4ea..58bd9dc6452 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1374,10 +1374,10 @@ UNIT_TEST_PROGRAMS += t-reftable-basics\n UNIT_TEST_PROGRAMS += t-reftable-block\n UNIT_TEST_PROGRAMS += t-reftable-merged\n UNIT_TEST_PROGRAMS += t-reftable-pq\n-UNIT_TEST_PROGRAMS += t-reftable-reader\n UNIT_TEST_PROGRAMS += t-reftable-readwrite\n UNIT_TEST_PROGRAMS += t-reftable-record\n UNIT_TEST_PROGRAMS += t-reftable-stack\n+UNIT_TEST_PROGRAMS += t-reftable-table\n UNIT_TEST_PROGRAMS += t-trailer\n UNIT_TEST_PROGRAMS += t-urlmatch-normalization\n UNIT_TEST_PROGS = $(patsubst %,$(UNIT_TEST_BIN)/%$X,$(UNIT_TEST_PROGRAMS))\n@@ -2729,10 +2729,10 @@ REFTABLE_OBJS += reftable/blocksource.o\n REFTABLE_OBJS += reftable/iter.o\n REFTABLE_OBJS += reftable/merged.o\n REFTABLE_OBJS += reftable/pq.o\n-REFTABLE_OBJS += reftable/reader.o\n REFTABLE_OBJS += reftable/record.o\n REFTABLE_OBJS += reftable/stack.o\n REFTABLE_OBJS += reftable/system.o\n+REFTABLE_OBJS += reftable/table.o\n REFTABLE_OBJS += reftable/tree.o\n REFTABLE_OBJS += reftable/writer.o\n \ndiff --git a/meson.build b/meson.build\nindex efe2871c9db..61986cc48db 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -410,10 +410,10 @@ libgit_sources = [\n   'reftable/iter.c',\n   'reftable/merged.c',\n   'reftable/pq.c',\n-  'reftable/reader.c',\n   'reftable/record.c',\n   'reftable/stack.c',\n   'reftable/system.c',\n+  'reftable/table.c',\n   'reftable/tree.c',\n   'reftable/writer.c',\n   'remote.c',\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex 44a155e756e..7376f263c99 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -12,8 +12,8 @@\n \n #include \"block.h\"\n #include \"constants.h\"\n-#include \"reader.h\"\n #include \"reftable-error.h\"\n+#include \"table.h\"\n \n int iterator_seek(struct reftable_iterator *it, struct reftable_record *want)\n {\n@@ -130,8 +130,8 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n \treftable_block_done(&it->block_reader.block);\n \n \toff = it->offsets[it->offset_idx++];\n-\terr = reader_init_block_reader(it->r, &it->block_reader, off,\n-\t\t\t\t       BLOCK_TYPE_REF);\n+\terr = table_init_block_reader(it->table, &it->block_reader, off,\n+\t\t\t\t      BLOCK_TYPE_REF);\n \tif (err < 0) {\n \t\treturn err;\n \t}\n@@ -181,7 +181,7 @@ static int indexed_table_ref_iter_next(void *p, struct reftable_record *rec)\n }\n \n int indexed_table_ref_iter_new(struct indexed_table_ref_iter **dest,\n-\t\t\t       struct reftable_reader *r, uint8_t *oid,\n+\t\t\t       struct reftable_table *t, uint8_t *oid,\n \t\t\t       int oid_len, uint64_t *offsets, int offset_len)\n {\n \tstruct indexed_table_ref_iter empty = INDEXED_TABLE_REF_ITER_INIT;\n@@ -195,7 +195,7 @@ int indexed_table_ref_iter_new(struct indexed_table_ref_iter **dest,\n \t}\n \n \t*itr = empty;\n-\titr->r = r;\n+\titr->table = t;\n \n \terr = reftable_buf_add(&itr->oid, oid, oid_len);\n \tif (err < 0)\ndiff --git a/reftable/iter.h b/reftable/iter.h\nindex bd217405f55..add5c9472fc 100644\n--- a/reftable/iter.h\n+++ b/reftable/iter.h\n@@ -59,7 +59,7 @@ void iterator_from_filtering_ref_iterator(struct reftable_iterator *,\n  * but using the object index.\n  */\n struct indexed_table_ref_iter {\n-\tstruct reftable_reader *r;\n+\tstruct reftable_table *table;\n \tstruct reftable_buf oid;\n \n \t/* mutable */\n@@ -83,7 +83,7 @@ void iterator_from_indexed_table_ref_iter(struct reftable_iterator *it,\n \n /* Takes ownership of `offsets` */\n int indexed_table_ref_iter_new(struct indexed_table_ref_iter **dest,\n-\t\t\t       struct reftable_reader *r, uint8_t *oid,\n+\t\t\t       struct reftable_table *t, uint8_t *oid,\n \t\t\t       int oid_len, uint64_t *offsets, int offset_len);\n \n #endif\ndiff --git a/reftable/merged.c b/reftable/merged.c\nindex 1829a081756..d5b974d660e 100644\n--- a/reftable/merged.c\n+++ b/reftable/merged.c\n@@ -11,11 +11,11 @@\n #include \"constants.h\"\n #include \"iter.h\"\n #include \"pq.h\"\n-#include \"reader.h\"\n #include \"record.h\"\n #include \"reftable-merged.h\"\n #include \"reftable-error.h\"\n #include \"system.h\"\n+#include \"table.h\"\n \n struct merged_subiter {\n \tstruct reftable_iterator iter;\n@@ -192,7 +192,7 @@ static void iterator_from_merged_iter(struct reftable_iterator *it,\n }\n \n int reftable_merged_table_new(struct reftable_merged_table **dest,\n-\t\t\t      struct reftable_reader **readers, size_t n,\n+\t\t\t      struct reftable_table **tables, size_t n,\n \t\t\t      enum reftable_hash hash_id)\n {\n \tstruct reftable_merged_table *m = NULL;\n@@ -200,10 +200,10 @@ int reftable_merged_table_new(struct reftable_merged_table **dest,\n \tuint64_t first_min = 0;\n \n \tfor (size_t i = 0; i < n; i++) {\n-\t\tuint64_t min = reftable_reader_min_update_index(readers[i]);\n-\t\tuint64_t max = reftable_reader_max_update_index(readers[i]);\n+\t\tuint64_t min = reftable_table_min_update_index(tables[i]);\n+\t\tuint64_t max = reftable_table_max_update_index(tables[i]);\n \n-\t\tif (reftable_reader_hash_id(readers[i]) != hash_id) {\n+\t\tif (reftable_table_hash_id(tables[i]) != hash_id) {\n \t\t\treturn REFTABLE_FORMAT_ERROR;\n \t\t}\n \t\tif (i == 0 || min < first_min) {\n@@ -218,8 +218,8 @@ int reftable_merged_table_new(struct reftable_merged_table **dest,\n \tif (!m)\n \t\treturn REFTABLE_OUT_OF_MEMORY_ERROR;\n \n-\tm->readers = readers;\n-\tm->readers_len = n;\n+\tm->tables = tables;\n+\tm->tables_len = n;\n \tm->min = first_min;\n \tm->max = last_max;\n \tm->hash_id = hash_id;\n@@ -254,20 +254,20 @@ int merged_table_init_iter(struct reftable_merged_table *mt,\n \tstruct merged_iter *mi = NULL;\n \tint ret;\n \n-\tif (mt->readers_len) {\n-\t\tREFTABLE_CALLOC_ARRAY(subiters, mt->readers_len);\n+\tif (mt->tables_len) {\n+\t\tREFTABLE_CALLOC_ARRAY(subiters, mt->tables_len);\n \t\tif (!subiters) {\n \t\t\tret = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\t\tgoto out;\n \t\t}\n \t}\n \n-\tfor (size_t i = 0; i < mt->readers_len; i++) {\n+\tfor (size_t i = 0; i < mt->tables_len; i++) {\n \t\tret = reftable_record_init(&subiters[i].rec, typ);\n \t\tif (ret < 0)\n \t\t\tgoto out;\n \n-\t\tret = reader_init_iter(mt->readers[i], &subiters[i].iter, typ);\n+\t\tret = table_init_iter(mt->tables[i], &subiters[i].iter, typ);\n \t\tif (ret < 0)\n \t\t\tgoto out;\n \t}\n@@ -280,14 +280,14 @@ int merged_table_init_iter(struct reftable_merged_table *mt,\n \tmi->advance_index = -1;\n \tmi->suppress_deletions = mt->suppress_deletions;\n \tmi->subiters = subiters;\n-\tmi->subiters_len = mt->readers_len;\n+\tmi->subiters_len = mt->tables_len;\n \n \titerator_from_merged_iter(it, mi);\n \tret = 0;\n \n out:\n \tif (ret < 0) {\n-\t\tfor (size_t i = 0; subiters && i < mt->readers_len; i++) {\n+\t\tfor (size_t i = 0; subiters && i < mt->tables_len; i++) {\n \t\t\treftable_iterator_destroy(&subiters[i].iter);\n \t\t\treftable_record_release(&subiters[i].rec);\n \t\t}\ndiff --git a/reftable/merged.h b/reftable/merged.h\nindex ae92409d342..4317e5f5f67 100644\n--- a/reftable/merged.h\n+++ b/reftable/merged.h\n@@ -13,8 +13,8 @@\n #include \"reftable-basics.h\"\n \n struct reftable_merged_table {\n-\tstruct reftable_reader **readers;\n-\tsize_t readers_len;\n+\tstruct reftable_table **tables;\n+\tsize_t tables_len;\n \tenum reftable_hash hash_id;\n \n \t/* If unset, produce deletions. This is useful for compaction. For the\ndiff --git a/reftable/reftable-merged.h b/reftable/reftable-merged.h\nindex 445756e475a..e5af846b32a 100644\n--- a/reftable/reftable-merged.h\n+++ b/reftable/reftable-merged.h\n@@ -26,14 +26,14 @@\n /* A merged table is implements seeking/iterating over a stack of tables. */\n struct reftable_merged_table;\n \n-struct reftable_reader;\n+struct reftable_table;\n \n /*\n- * reftable_merged_table_new creates a new merged table. The readers must be\n+ * reftable_merged_table_new creates a new merged table. The tables must be\n  * kept alive as long as the merged table is still in use.\n  */\n int reftable_merged_table_new(struct reftable_merged_table **dest,\n-\t\t\t      struct reftable_reader **readers, size_t n,\n+\t\t\t      struct reftable_table **tables, size_t n,\n \t\t\t      enum reftable_hash hash_id);\n \n /* Initialize a merged table iterator for reading refs. */\ndiff --git a/reftable/reftable-reader.h b/reftable/reftable-reader.h\ndeleted file mode 100644\nindex 9b6a8ac0716..00000000000\n--- a/reftable/reftable-reader.h\n+++ /dev/null\n@@ -1,72 +0,0 @@\n-/*\n- * Copyright 2020 Google LLC\n- *\n- * Use of this source code is governed by a BSD-style\n- * license that can be found in the LICENSE file or at\n- * https://developers.google.com/open-source/licenses/bsd\n- */\n-\n-#ifndef REFTABLE_READER_H\n-#define REFTABLE_READER_H\n-\n-#include \"reftable-iterator.h\"\n-#include \"reftable-blocksource.h\"\n-\n-/*\n- * Reading single tables\n- *\n- * The follow routines are for reading single files. For an\n- * application-level interface, skip ahead to struct\n- * reftable_merged_table and struct reftable_stack.\n- */\n-\n-/* The reader struct is a handle to an open reftable file. */\n-struct reftable_reader;\n-\n-/* reftable_reader_new opens a reftable for reading. If successful,\n- * returns 0 code and sets pp. The name is used for creating a\n- * stack. Typically, it is the basename of the file. The block source\n- * `src` is owned by the reader, and is closed on calling\n- * reftable_reader_destroy(). On error, the block source `src` is\n- * closed as well.\n- */\n-int reftable_reader_new(struct reftable_reader **pp,\n-\t\t\tstruct reftable_block_source *src, const char *name);\n-\n-/*\n- * Manage the reference count of the reftable reader. A newly initialized\n- * reader starts with a refcount of 1 and will be deleted once the refcount has\n- * reached 0.\n- *\n- * This is required because readers may have longer lifetimes than the stack\n- * they belong to. The stack may for example be reloaded while the old tables\n- * are still being accessed by an iterator.\n- */\n-void reftable_reader_incref(struct reftable_reader *reader);\n-void reftable_reader_decref(struct reftable_reader *reader);\n-\n-/* Initialize a reftable iterator for reading refs. */\n-int reftable_reader_init_ref_iterator(struct reftable_reader *r,\n-\t\t\t\t      struct reftable_iterator *it);\n-\n-/* Initialize a reftable iterator for reading logs. */\n-int reftable_reader_init_log_iterator(struct reftable_reader *r,\n-\t\t\t\t      struct reftable_iterator *it);\n-\n-/* returns the hash ID used in this table. */\n-enum reftable_hash reftable_reader_hash_id(struct reftable_reader *r);\n-\n-/* return an iterator for the refs pointing to `oid`. */\n-int reftable_reader_refs_for(struct reftable_reader *r,\n-\t\t\t     struct reftable_iterator *it, uint8_t *oid);\n-\n-/* return the max_update_index for a table */\n-uint64_t reftable_reader_max_update_index(struct reftable_reader *r);\n-\n-/* return the min_update_index for a table */\n-uint64_t reftable_reader_min_update_index(struct reftable_reader *r);\n-\n-/* print blocks onto stdout for debugging. */\n-int reftable_reader_print_blocks(const char *tablename);\n-\n-#endif\ndiff --git a/reftable/reftable-table.h b/reftable/reftable-table.h\nnew file mode 100644\nindex 00000000000..94379026727\n--- /dev/null\n+++ b/reftable/reftable-table.h\n@@ -0,0 +1,72 @@\n+/*\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n+\n+#ifndef REFTABLE_TABLE_H\n+#define REFTABLE_TABLE_H\n+\n+#include \"reftable-iterator.h\"\n+#include \"reftable-blocksource.h\"\n+\n+/*\n+ * Reading single tables\n+ *\n+ * The follow routines are for reading single files. For an\n+ * application-level interface, skip ahead to struct\n+ * reftable_merged_table and struct reftable_stack.\n+ */\n+\n+/* The table struct is a handle to an open reftable file. */\n+struct reftable_table;\n+\n+/* reftable_table_new opens a reftable for reading. If successful,\n+ * returns 0 code and sets pp. The name is used for creating a\n+ * stack. Typically, it is the basename of the file. The block source\n+ * `src` is owned by the table, and is closed on calling\n+ * reftable_table_destroy(). On error, the block source `src` is\n+ * closed as well.\n+ */\n+int reftable_table_new(struct reftable_table **out,\n+\t\t       struct reftable_block_source *src, const char *name);\n+\n+/*\n+ * Manage the reference count of the reftable table. A newly initialized\n+ * table starts with a refcount of 1 and will be deleted once the refcount has\n+ * reached 0.\n+ *\n+ * This is required because tables may have longer lifetimes than the stack\n+ * they belong to. The stack may for example be reloaded while the old tables\n+ * are still being accessed by an iterator.\n+ */\n+void reftable_table_incref(struct reftable_table *table);\n+void reftable_table_decref(struct reftable_table *table);\n+\n+/* Initialize a reftable iterator for reading refs. */\n+int reftable_table_init_ref_iterator(struct reftable_table *t,\n+\t\t\t\t     struct reftable_iterator *it);\n+\n+/* Initialize a reftable iterator for reading logs. */\n+int reftable_table_init_log_iterator(struct reftable_table *t,\n+\t\t\t\t     struct reftable_iterator *it);\n+\n+/* returns the hash ID used in this table. */\n+enum reftable_hash reftable_table_hash_id(struct reftable_table *t);\n+\n+/* return an iterator for the refs pointing to `oid`. */\n+int reftable_table_refs_for(struct reftable_table *t,\n+\t\t\t    struct reftable_iterator *it, uint8_t *oid);\n+\n+/* return the max_update_index for a table */\n+uint64_t reftable_table_max_update_index(struct reftable_table *t);\n+\n+/* return the min_update_index for a table */\n+uint64_t reftable_table_min_update_index(struct reftable_table *t);\n+\n+/* print blocks onto stdout for debugging. */\n+int reftable_table_print_blocks(const char *tablename);\n+\n+#endif\ndiff --git a/reftable/stack.c b/reftable/stack.c\nindex bc3cfa91170..cc48e725b14 100644\n--- a/reftable/stack.c\n+++ b/reftable/stack.c\n@@ -11,10 +11,10 @@\n #include \"system.h\"\n #include \"constants.h\"\n #include \"merged.h\"\n-#include \"reader.h\"\n #include \"reftable-error.h\"\n #include \"reftable-record.h\"\n #include \"reftable-merged.h\"\n+#include \"table.h\"\n #include \"writer.h\"\n \n static int stack_try_add(struct reftable_stack *st,\n@@ -248,11 +248,11 @@ void reftable_stack_destroy(struct reftable_stack *st)\n \t\tREFTABLE_FREE_AND_NULL(names);\n \t}\n \n-\tif (st->readers) {\n+\tif (st->tables) {\n \t\tstruct reftable_buf filename = REFTABLE_BUF_INIT;\n \n-\t\tfor (size_t i = 0; i < st->readers_len; i++) {\n-\t\t\tconst char *name = reader_name(st->readers[i]);\n+\t\tfor (size_t i = 0; i < st->tables_len; i++) {\n+\t\t\tconst char *name = reftable_table_name(st->tables[i]);\n \t\t\tint try_unlinking = 1;\n \n \t\t\treftable_buf_reset(&filename);\n@@ -260,7 +260,7 @@ void reftable_stack_destroy(struct reftable_stack *st)\n \t\t\t\tif (stack_filename(&filename, st, name) < 0)\n \t\t\t\t\ttry_unlinking = 0;\n \t\t\t}\n-\t\t\treftable_reader_decref(st->readers[i]);\n+\t\t\treftable_table_decref(st->tables[i]);\n \n \t\t\tif (try_unlinking && filename.len) {\n \t\t\t\t/* On Windows, can only unlink after closing. */\n@@ -269,8 +269,8 @@ void reftable_stack_destroy(struct reftable_stack *st)\n \t\t}\n \n \t\treftable_buf_release(&filename);\n-\t\tst->readers_len = 0;\n-\t\tREFTABLE_FREE_AND_NULL(st->readers);\n+\t\tst->tables_len = 0;\n+\t\tREFTABLE_FREE_AND_NULL(st->tables);\n \t}\n \n \tif (st->list_fd >= 0) {\n@@ -284,14 +284,14 @@ void reftable_stack_destroy(struct reftable_stack *st)\n \tfree_names(names);\n }\n \n-static struct reftable_reader **stack_copy_readers(struct reftable_stack *st,\n-\t\t\t\t\t\t   size_t cur_len)\n+static struct reftable_table **stack_copy_tables(struct reftable_stack *st,\n+\t\t\t\t\t\t size_t cur_len)\n {\n-\tstruct reftable_reader **cur = reftable_calloc(cur_len, sizeof(*cur));\n+\tstruct reftable_table **cur = reftable_calloc(cur_len, sizeof(*cur));\n \tif (!cur)\n \t\treturn NULL;\n \tfor (size_t i = 0; i < cur_len; i++)\n-\t\tcur[i] = st->readers[i];\n+\t\tcur[i] = st->tables[i];\n \treturn cur;\n }\n \n@@ -299,19 +299,19 @@ static int reftable_stack_reload_once(struct reftable_stack *st,\n \t\t\t\t      const char **names,\n \t\t\t\t      int reuse_open)\n {\n-\tsize_t cur_len = !st->merged ? 0 : st->merged->readers_len;\n-\tstruct reftable_reader **cur = NULL;\n-\tstruct reftable_reader **reused = NULL;\n-\tstruct reftable_reader **new_readers = NULL;\n+\tsize_t cur_len = !st->merged ? 0 : st->merged->tables_len;\n+\tstruct reftable_table **cur = NULL;\n+\tstruct reftable_table **reused = NULL;\n+\tstruct reftable_table **new_tables = NULL;\n \tsize_t reused_len = 0, reused_alloc = 0, names_len;\n-\tsize_t new_readers_len = 0;\n+\tsize_t new_tables_len = 0;\n \tstruct reftable_merged_table *new_merged = NULL;\n \tstruct reftable_buf table_path = REFTABLE_BUF_INIT;\n \tint err = 0;\n \tsize_t i;\n \n \tif (cur_len) {\n-\t\tcur = stack_copy_readers(st, cur_len);\n+\t\tcur = stack_copy_tables(st, cur_len);\n \t\tif (!cur) {\n \t\t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\t\tgoto done;\n@@ -321,28 +321,28 @@ static int reftable_stack_reload_once(struct reftable_stack *st,\n \tnames_len = names_length(names);\n \n \tif (names_len) {\n-\t\tnew_readers = reftable_calloc(names_len, sizeof(*new_readers));\n-\t\tif (!new_readers) {\n+\t\tnew_tables = reftable_calloc(names_len, sizeof(*new_tables));\n+\t\tif (!new_tables) {\n \t\t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\t\tgoto done;\n \t\t}\n \t}\n \n \twhile (*names) {\n-\t\tstruct reftable_reader *rd = NULL;\n+\t\tstruct reftable_table *table = NULL;\n \t\tconst char *name = *names++;\n \n \t\t/* this is linear; we assume compaction keeps the number of\n \t\t   tables under control so this is not quadratic. */\n \t\tfor (i = 0; reuse_open && i < cur_len; i++) {\n \t\t\tif (cur[i] && 0 == strcmp(cur[i]->name, name)) {\n-\t\t\t\trd = cur[i];\n+\t\t\t\ttable = cur[i];\n \t\t\t\tcur[i] = NULL;\n \n \t\t\t\t/*\n \t\t\t\t * When reloading the stack fails, we end up\n-\t\t\t\t * releasing all new readers. This also\n-\t\t\t\t * includes the reused readers, even though\n+\t\t\t\t * releasing all new tables. This also\n+\t\t\t\t * includes the reused tables, even though\n \t\t\t\t * they are still in used by the old stack. We\n \t\t\t\t * thus need to keep them alive here, which we\n \t\t\t\t * do by bumping their refcount.\n@@ -354,13 +354,13 @@ static int reftable_stack_reload_once(struct reftable_stack *st,\n \t\t\t\t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\t\t\t\tgoto done;\n \t\t\t\t}\n-\t\t\t\treused[reused_len++] = rd;\n-\t\t\t\treftable_reader_incref(rd);\n+\t\t\t\treused[reused_len++] = table;\n+\t\t\t\treftable_table_incref(table);\n \t\t\t\tbreak;\n \t\t\t}\n \t\t}\n \n-\t\tif (!rd) {\n+\t\tif (!table) {\n \t\t\tstruct reftable_block_source src = { NULL };\n \n \t\t\terr = stack_filename(&table_path, st, name);\n@@ -372,36 +372,36 @@ static int reftable_stack_reload_once(struct reftable_stack *st,\n \t\t\tif (err < 0)\n \t\t\t\tgoto done;\n \n-\t\t\terr = reftable_reader_new(&rd, &src, name);\n+\t\t\terr = reftable_table_new(&table, &src, name);\n \t\t\tif (err < 0)\n \t\t\t\tgoto done;\n \t\t}\n \n-\t\tnew_readers[new_readers_len] = rd;\n-\t\tnew_readers_len++;\n+\t\tnew_tables[new_tables_len] = table;\n+\t\tnew_tables_len++;\n \t}\n \n \t/* success! */\n-\terr = reftable_merged_table_new(&new_merged, new_readers,\n-\t\t\t\t\tnew_readers_len, st->opts.hash_id);\n+\terr = reftable_merged_table_new(&new_merged, new_tables,\n+\t\t\t\t\tnew_tables_len, st->opts.hash_id);\n \tif (err < 0)\n \t\tgoto done;\n \n \t/*\n-\t * Close the old, non-reused readers and proactively try to unlink\n+\t * Close the old, non-reused tables and proactively try to unlink\n \t * them. This is done for systems like Windows, where the underlying\n-\t * file of such an open reader wouldn't have been possible to be\n+\t * file of such an open table wouldn't have been possible to be\n \t * unlinked by the compacting process.\n \t */\n \tfor (i = 0; i < cur_len; i++) {\n \t\tif (cur[i]) {\n-\t\t\tconst char *name = reader_name(cur[i]);\n+\t\t\tconst char *name = reftable_table_name(cur[i]);\n \n \t\t\terr = stack_filename(&table_path, st, name);\n \t\t\tif (err < 0)\n \t\t\t\tgoto done;\n \n-\t\t\treftable_reader_decref(cur[i]);\n+\t\t\treftable_table_decref(cur[i]);\n \t\t\tunlink(table_path.buf);\n \t\t}\n \t}\n@@ -412,25 +412,25 @@ static int reftable_stack_reload_once(struct reftable_stack *st,\n \tnew_merged->suppress_deletions = 1;\n \tst->merged = new_merged;\n \n-\tif (st->readers)\n-\t\treftable_free(st->readers);\n-\tst->readers = new_readers;\n-\tst->readers_len = new_readers_len;\n-\tnew_readers = NULL;\n-\tnew_readers_len = 0;\n+\tif (st->tables)\n+\t\treftable_free(st->tables);\n+\tst->tables = new_tables;\n+\tst->tables_len = new_tables_len;\n+\tnew_tables = NULL;\n+\tnew_tables_len = 0;\n \n \t/*\n-\t * Decrement the refcount of reused readers again. This only needs to\n+\t * Decrement the refcount of reused tables again. This only needs to\n \t * happen on the successful case, because on the unsuccessful one we\n-\t * decrement their refcount via `new_readers`.\n+\t * decrement their refcount via `new_tables`.\n \t */\n \tfor (i = 0; i < reused_len; i++)\n-\t\treftable_reader_decref(reused[i]);\n+\t\treftable_table_decref(reused[i]);\n \n done:\n-\tfor (i = 0; i < new_readers_len; i++)\n-\t\treftable_reader_decref(new_readers[i]);\n-\treftable_free(new_readers);\n+\tfor (i = 0; i < new_tables_len; i++)\n+\t\treftable_table_decref(new_tables[i]);\n+\treftable_free(new_tables);\n \treftable_free(reused);\n \treftable_free(cur);\n \treftable_buf_release(&table_path);\n@@ -615,10 +615,10 @@ static int stack_uptodate(struct reftable_stack *st)\n \t\t\t/*\n \t\t\t * It's fine for \"tables.list\" to not exist. In that\n \t\t\t * case, we have to refresh when the loaded stack has\n-\t\t\t * any readers.\n+\t\t\t * any tables.\n \t\t\t */\n \t\t\tif (errno == ENOENT)\n-\t\t\t\treturn !!st->readers_len;\n+\t\t\t\treturn !!st->tables_len;\n \t\t\treturn REFTABLE_IO_ERROR;\n \t\t}\n \n@@ -637,19 +637,19 @@ static int stack_uptodate(struct reftable_stack *st)\n \tif (err < 0)\n \t\treturn err;\n \n-\tfor (size_t i = 0; i < st->readers_len; i++) {\n+\tfor (size_t i = 0; i < st->tables_len; i++) {\n \t\tif (!names[i]) {\n \t\t\terr = 1;\n \t\t\tgoto done;\n \t\t}\n \n-\t\tif (strcmp(st->readers[i]->name, names[i])) {\n+\t\tif (strcmp(st->tables[i]->name, names[i])) {\n \t\t\terr = 1;\n \t\t\tgoto done;\n \t\t}\n \t}\n \n-\tif (names[st->merged->readers_len]) {\n+\tif (names[st->merged->tables_len]) {\n \t\terr = 1;\n \t\tgoto done;\n \t}\n@@ -792,8 +792,8 @@ int reftable_addition_commit(struct reftable_addition *add)\n \tif (add->new_tables_len == 0)\n \t\tgoto done;\n \n-\tfor (i = 0; i < add->stack->merged->readers_len; i++) {\n-\t\tif ((err = reftable_buf_addstr(&table_list, add->stack->readers[i]->name)) < 0 ||\n+\tfor (i = 0; i < add->stack->merged->tables_len; i++) {\n+\t\tif ((err = reftable_buf_addstr(&table_list, add->stack->tables[i]->name)) < 0 ||\n \t\t    (err = reftable_buf_addstr(&table_list, \"\\n\")) < 0)\n \t\t\tgoto done;\n \t}\n@@ -1000,9 +1000,9 @@ int reftable_addition_add(struct reftable_addition *add,\n \n uint64_t reftable_stack_next_update_index(struct reftable_stack *st)\n {\n-\tint sz = st->merged->readers_len;\n+\tint sz = st->merged->tables_len;\n \tif (sz > 0)\n-\t\treturn reftable_reader_max_update_index(st->readers[sz - 1]) +\n+\t\treturn reftable_table_max_update_index(st->tables[sz - 1]) +\n \t\t       1;\n \treturn 1;\n }\n@@ -1021,8 +1021,8 @@ static int stack_compact_locked(struct reftable_stack *st,\n \tstruct reftable_tmpfile tab_file = REFTABLE_TMPFILE_INIT;\n \tint err = 0;\n \n-\terr = format_name(&next_name, reftable_reader_min_update_index(st->readers[first]),\n-\t\t\t  reftable_reader_max_update_index(st->readers[last]));\n+\terr = format_name(&next_name, reftable_table_min_update_index(st->tables[first]),\n+\t\t\t  reftable_table_max_update_index(st->tables[last]));\n \tif (err < 0)\n \t\tgoto done;\n \n@@ -1087,13 +1087,13 @@ static int stack_write_compact(struct reftable_stack *st,\n \tint err = 0;\n \n \tfor (size_t i = first; i <= last; i++)\n-\t\tst->stats.bytes += st->readers[i]->size;\n-\terr = reftable_writer_set_limits(wr, st->readers[first]->min_update_index,\n-\t\t\t\t\t st->readers[last]->max_update_index);\n+\t\tst->stats.bytes += st->tables[i]->size;\n+\terr = reftable_writer_set_limits(wr, st->tables[first]->min_update_index,\n+\t\t\t\t\t st->tables[last]->max_update_index);\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = reftable_merged_table_new(&mt, st->readers + first, subtabs_len,\n+\terr = reftable_merged_table_new(&mt, st->tables + first, subtabs_len,\n \t\t\t\t\tst->opts.hash_id);\n \tif (err < 0)\n \t\tgoto done;\n@@ -1250,7 +1250,7 @@ static int stack_compact_range(struct reftable_stack *st,\n \t\ttable_locks[i] = REFTABLE_FLOCK_INIT;\n \n \tfor (i = last + 1; i > first; i--) {\n-\t\terr = stack_filename(&table_name, st, reader_name(st->readers[i - 1]));\n+\t\terr = stack_filename(&table_name, st, reftable_table_name(st->tables[i - 1]));\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n@@ -1376,7 +1376,7 @@ static int stack_compact_range(struct reftable_stack *st,\n \t\t * compacted in the updated \"tables.list\" file.\n \t\t */\n \t\tfor (size_t i = 0; names[i]; i++) {\n-\t\t\tif (strcmp(names[i], st->readers[first]->name))\n+\t\t\tif (strcmp(names[i], st->tables[first]->name))\n \t\t\t\tcontinue;\n \n \t\t\t/*\n@@ -1386,8 +1386,8 @@ static int stack_compact_range(struct reftable_stack *st,\n \t\t\t * have compacted them.\n \t\t\t */\n \t\t\tfor (size_t j = 1; j < last - first + 1; j++) {\n-\t\t\t\tconst char *old = first + j < st->merged->readers_len ?\n-\t\t\t\t\tst->readers[first + j]->name : NULL;\n+\t\t\t\tconst char *old = first + j < st->merged->tables_len ?\n+\t\t\t\t\tst->tables[first + j]->name : NULL;\n \t\t\t\tconst char *new = names[i + j];\n \n \t\t\t\t/*\n@@ -1427,16 +1427,16 @@ static int stack_compact_range(struct reftable_stack *st,\n \t\t * `fd_read_lines()` uses a `NULL` sentinel to indicate that\n \t\t * the array is at its end. As we use `free_names()` to free\n \t\t * the array, we need to include this sentinel value here and\n-\t\t * thus have to allocate `readers_len + 1` many entries.\n+\t\t * thus have to allocate `tables_len + 1` many entries.\n \t\t */\n-\t\tREFTABLE_CALLOC_ARRAY(names, st->merged->readers_len + 1);\n+\t\tREFTABLE_CALLOC_ARRAY(names, st->merged->tables_len + 1);\n \t\tif (!names) {\n \t\t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\t\tgoto done;\n \t\t}\n \n-\t\tfor (size_t i = 0; i < st->merged->readers_len; i++) {\n-\t\t\tnames[i] = reftable_strdup(st->readers[i]->name);\n+\t\tfor (size_t i = 0; i < st->merged->tables_len; i++) {\n+\t\t\tnames[i] = reftable_strdup(st->tables[i]->name);\n \t\t\tif (!names[i]) {\n \t\t\t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\t\t\tgoto done;\n@@ -1451,8 +1451,8 @@ static int stack_compact_range(struct reftable_stack *st,\n \t * it into place now.\n \t */\n \tif (!is_empty_table) {\n-\t\terr = format_name(&new_table_name, st->readers[first]->min_update_index,\n-\t\t\t\t  st->readers[last]->max_update_index);\n+\t\terr = format_name(&new_table_name, st->tables[first]->min_update_index,\n+\t\t\t\t  st->tables[last]->max_update_index);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n@@ -1559,7 +1559,7 @@ static int stack_compact_range(struct reftable_stack *st,\n int reftable_stack_compact_all(struct reftable_stack *st,\n \t\t\t       struct reftable_log_expiry_config *config)\n {\n-\tsize_t last = st->merged->readers_len ? st->merged->readers_len - 1 : 0;\n+\tsize_t last = st->merged->tables_len ? st->merged->tables_len - 1 : 0;\n \treturn stack_compact_range(st, 0, last, config, 0);\n }\n \n@@ -1650,12 +1650,12 @@ static uint64_t *stack_table_sizes_for_compaction(struct reftable_stack *st)\n \tint overhead = header_size(version) - 1;\n \tuint64_t *sizes;\n \n-\tREFTABLE_CALLOC_ARRAY(sizes, st->merged->readers_len);\n+\tREFTABLE_CALLOC_ARRAY(sizes, st->merged->tables_len);\n \tif (!sizes)\n \t\treturn NULL;\n \n-\tfor (size_t i = 0; i < st->merged->readers_len; i++)\n-\t\tsizes[i] = st->readers[i]->size - overhead;\n+\tfor (size_t i = 0; i < st->merged->tables_len; i++)\n+\t\tsizes[i] = st->tables[i]->size - overhead;\n \n \treturn sizes;\n }\n@@ -1665,14 +1665,14 @@ int reftable_stack_auto_compact(struct reftable_stack *st)\n \tstruct segment seg;\n \tuint64_t *sizes;\n \n-\tif (st->merged->readers_len < 2)\n+\tif (st->merged->tables_len < 2)\n \t\treturn 0;\n \n \tsizes = stack_table_sizes_for_compaction(st);\n \tif (!sizes)\n \t\treturn REFTABLE_OUT_OF_MEMORY_ERROR;\n \n-\tseg = suggest_compaction_segment(sizes, st->merged->readers_len,\n+\tseg = suggest_compaction_segment(sizes, st->merged->tables_len,\n \t\t\t\t\t st->opts.auto_compaction_factor);\n \treftable_free(sizes);\n \n@@ -1763,7 +1763,7 @@ static void remove_maybe_stale_table(struct reftable_stack *st, uint64_t max,\n \tint err = 0;\n \tuint64_t update_idx = 0;\n \tstruct reftable_block_source src = { NULL };\n-\tstruct reftable_reader *rd = NULL;\n+\tstruct reftable_table *table = NULL;\n \tstruct reftable_buf table_path = REFTABLE_BUF_INIT;\n \n \terr = stack_filename(&table_path, st, name);\n@@ -1774,12 +1774,12 @@ static void remove_maybe_stale_table(struct reftable_stack *st, uint64_t max,\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = reftable_reader_new(&rd, &src, name);\n+\terr = reftable_table_new(&table, &src, name);\n \tif (err < 0)\n \t\tgoto done;\n \n-\tupdate_idx = reftable_reader_max_update_index(rd);\n-\treftable_reader_decref(rd);\n+\tupdate_idx = reftable_table_max_update_index(table);\n+\treftable_table_decref(table);\n \n \tif (update_idx <= max) {\n \t\tunlink(table_path.buf);\n@@ -1803,8 +1803,8 @@ static int reftable_stack_clean_locked(struct reftable_stack *st)\n \t\tif (!is_table_name(d->d_name))\n \t\t\tcontinue;\n \n-\t\tfor (size_t i = 0; !found && i < st->readers_len; i++)\n-\t\t\tfound = !strcmp(reader_name(st->readers[i]), d->d_name);\n+\t\tfor (size_t i = 0; !found && i < st->tables_len; i++)\n+\t\t\tfound = !strcmp(reftable_table_name(st->tables[i]), d->d_name);\n \t\tif (found)\n \t\t\tcontinue;\n \ndiff --git a/reftable/stack.h b/reftable/stack.h\nindex d91f5ce6072..bc28f2998ac 100644\n--- a/reftable/stack.h\n+++ b/reftable/stack.h\n@@ -22,8 +22,8 @@ struct reftable_stack {\n \n \tstruct reftable_write_options opts;\n \n-\tstruct reftable_reader **readers;\n-\tsize_t readers_len;\n+\tstruct reftable_table **tables;\n+\tsize_t tables_len;\n \tstruct reftable_merged_table *merged;\n \tstruct reftable_compaction_stats stats;\n };\ndiff --git a/reftable/reader.c b/reftable/table.c\nsimilarity index 73%\nrename from reftable/reader.c\nrename to reftable/table.c\nindex 155a26c30ff..440fb559ad1 100644\n--- a/reftable/reader.c\n+++ b/reftable/table.c\n@@ -6,7 +6,7 @@\n  * https://developers.google.com/open-source/licenses/bsd\n  */\n \n-#include \"reader.h\"\n+#include \"table.h\"\n \n #include \"system.h\"\n #include \"block.h\"\n@@ -39,48 +39,48 @@ void block_source_close(struct reftable_block_source *source)\n \tsource->ops = NULL;\n }\n \n-static struct reftable_reader_offsets *\n-reader_offsets_for(struct reftable_reader *r, uint8_t typ)\n+static struct reftable_table_offsets *\n+table_offsets_for(struct reftable_table *t, uint8_t typ)\n {\n \tswitch (typ) {\n \tcase BLOCK_TYPE_REF:\n-\t\treturn &r->ref_offsets;\n+\t\treturn &t->ref_offsets;\n \tcase BLOCK_TYPE_LOG:\n-\t\treturn &r->log_offsets;\n+\t\treturn &t->log_offsets;\n \tcase BLOCK_TYPE_OBJ:\n-\t\treturn &r->obj_offsets;\n+\t\treturn &t->obj_offsets;\n \t}\n \tabort();\n }\n \n-static int reader_get_block(struct reftable_reader *r,\n-\t\t\t    struct reftable_block *dest, uint64_t off,\n-\t\t\t    uint32_t sz)\n+static int table_get_block(struct reftable_table *t,\n+\t\t\t   struct reftable_block *dest, uint64_t off,\n+\t\t\t   uint32_t sz)\n {\n \tssize_t bytes_read;\n-\tif (off >= r->size)\n+\tif (off >= t->size)\n \t\treturn 0;\n-\tif (off + sz > r->size)\n-\t\tsz = r->size - off;\n+\tif (off + sz > t->size)\n+\t\tsz = t->size - off;\n \n-\tbytes_read = block_source_read_block(&r->source, dest, off, sz);\n+\tbytes_read = block_source_read_block(&t->source, dest, off, sz);\n \tif (bytes_read < 0)\n \t\treturn (int)bytes_read;\n \n \treturn 0;\n }\n \n-enum reftable_hash reftable_reader_hash_id(struct reftable_reader *r)\n+enum reftable_hash reftable_table_hash_id(struct reftable_table *t)\n {\n-\treturn r->hash_id;\n+\treturn t->hash_id;\n }\n \n-const char *reader_name(struct reftable_reader *r)\n+const char *reftable_table_name(struct reftable_table *t)\n {\n-\treturn r->name;\n+\treturn t->name;\n }\n \n-static int parse_footer(struct reftable_reader *r, uint8_t *footer,\n+static int parse_footer(struct reftable_table *t, uint8_t *footer,\n \t\t\tuint8_t *header)\n {\n \tuint8_t *f = footer;\n@@ -95,29 +95,29 @@ static int parse_footer(struct reftable_reader *r, uint8_t *footer,\n \t}\n \tf += 4;\n \n-\tif (memcmp(footer, header, header_size(r->version))) {\n+\tif (memcmp(footer, header, header_size(t->version))) {\n \t\terr = REFTABLE_FORMAT_ERROR;\n \t\tgoto done;\n \t}\n \n \tf++;\n-\tr->block_size = reftable_get_be24(f);\n+\tt->block_size = reftable_get_be24(f);\n \n \tf += 3;\n-\tr->min_update_index = reftable_get_be64(f);\n+\tt->min_update_index = reftable_get_be64(f);\n \tf += 8;\n-\tr->max_update_index = reftable_get_be64(f);\n+\tt->max_update_index = reftable_get_be64(f);\n \tf += 8;\n \n-\tif (r->version == 1) {\n-\t\tr->hash_id = REFTABLE_HASH_SHA1;\n+\tif (t->version == 1) {\n+\t\tt->hash_id = REFTABLE_HASH_SHA1;\n \t} else {\n \t\tswitch (reftable_get_be32(f)) {\n \t\tcase REFTABLE_FORMAT_ID_SHA1:\n-\t\t\tr->hash_id = REFTABLE_HASH_SHA1;\n+\t\t\tt->hash_id = REFTABLE_HASH_SHA1;\n \t\t\tbreak;\n \t\tcase REFTABLE_FORMAT_ID_SHA256:\n-\t\t\tr->hash_id = REFTABLE_HASH_SHA256;\n+\t\t\tt->hash_id = REFTABLE_HASH_SHA256;\n \t\t\tbreak;\n \t\tdefault:\n \t\t\terr = REFTABLE_FORMAT_ERROR;\n@@ -127,20 +127,20 @@ static int parse_footer(struct reftable_reader *r, uint8_t *footer,\n \t\tf += 4;\n \t}\n \n-\tr->ref_offsets.index_offset = reftable_get_be64(f);\n+\tt->ref_offsets.index_offset = reftable_get_be64(f);\n \tf += 8;\n \n-\tr->obj_offsets.offset = reftable_get_be64(f);\n+\tt->obj_offsets.offset = reftable_get_be64(f);\n \tf += 8;\n \n-\tr->object_id_len = r->obj_offsets.offset & ((1 << 5) - 1);\n-\tr->obj_offsets.offset >>= 5;\n+\tt->object_id_len = t->obj_offsets.offset & ((1 << 5) - 1);\n+\tt->obj_offsets.offset >>= 5;\n \n-\tr->obj_offsets.index_offset = reftable_get_be64(f);\n+\tt->obj_offsets.index_offset = reftable_get_be64(f);\n \tf += 8;\n-\tr->log_offsets.offset = reftable_get_be64(f);\n+\tt->log_offsets.offset = reftable_get_be64(f);\n \tf += 8;\n-\tr->log_offsets.index_offset = reftable_get_be64(f);\n+\tt->log_offsets.index_offset = reftable_get_be64(f);\n \tf += 8;\n \n \tcomputed_crc = crc32(0, footer, f - footer);\n@@ -151,13 +151,13 @@ static int parse_footer(struct reftable_reader *r, uint8_t *footer,\n \t\tgoto done;\n \t}\n \n-\tfirst_block_typ = header[header_size(r->version)];\n-\tr->ref_offsets.is_present = (first_block_typ == BLOCK_TYPE_REF);\n-\tr->ref_offsets.offset = 0;\n-\tr->log_offsets.is_present = (first_block_typ == BLOCK_TYPE_LOG ||\n-\t\t\t\t     r->log_offsets.offset > 0);\n-\tr->obj_offsets.is_present = r->obj_offsets.offset > 0;\n-\tif (r->obj_offsets.is_present && !r->object_id_len) {\n+\tfirst_block_typ = header[header_size(t->version)];\n+\tt->ref_offsets.is_present = (first_block_typ == BLOCK_TYPE_REF);\n+\tt->ref_offsets.offset = 0;\n+\tt->log_offsets.is_present = (first_block_typ == BLOCK_TYPE_LOG ||\n+\t\t\t\t     t->log_offsets.offset > 0);\n+\tt->obj_offsets.is_present = t->obj_offsets.offset > 0;\n+\tif (t->obj_offsets.is_present && !t->object_id_len) {\n \t\terr = REFTABLE_FORMAT_ERROR;\n \t\tgoto done;\n \t}\n@@ -168,7 +168,7 @@ static int parse_footer(struct reftable_reader *r, uint8_t *footer,\n }\n \n struct table_iter {\n-\tstruct reftable_reader *r;\n+\tstruct reftable_table *table;\n \tuint8_t typ;\n \tuint64_t block_off;\n \tstruct block_reader br;\n@@ -176,12 +176,12 @@ struct table_iter {\n \tint is_finished;\n };\n \n-static int table_iter_init(struct table_iter *ti, struct reftable_reader *r)\n+static int table_iter_init(struct table_iter *ti, struct reftable_table *t)\n {\n \tstruct block_iter bi = BLOCK_ITER_INIT;\n \tmemset(ti, 0, sizeof(*ti));\n-\treftable_reader_incref(r);\n-\tti->r = r;\n+\treftable_table_incref(t);\n+\tti->table = t;\n \tti->bi = bi;\n \treturn 0;\n }\n@@ -191,7 +191,7 @@ static int table_iter_next_in_block(struct table_iter *ti,\n {\n \tint res = block_iter_next(&ti->bi, rec);\n \tif (res == 0 && reftable_record_type(rec) == BLOCK_TYPE_REF) {\n-\t\trec->u.ref.update_index += ti->r->min_update_index;\n+\t\trec->u.ref.update_index += ti->table->min_update_index;\n \t}\n \n \treturn res;\n@@ -219,26 +219,26 @@ static int32_t extract_block_size(uint8_t *data, uint8_t *typ, uint64_t off,\n \treturn result;\n }\n \n-int reader_init_block_reader(struct reftable_reader *r, struct block_reader *br,\n-\t\t\t     uint64_t next_off, uint8_t want_typ)\n+int table_init_block_reader(struct reftable_table *t, struct block_reader *br,\n+\t\t\t    uint64_t next_off, uint8_t want_typ)\n {\n-\tint32_t guess_block_size = r->block_size ? r->block_size :\n+\tint32_t guess_block_size = t->block_size ? t->block_size :\n \t\t\t\t\t\t\t DEFAULT_BLOCK_SIZE;\n \tstruct reftable_block block = { NULL };\n \tuint8_t block_typ = 0;\n \tint err = 0;\n-\tuint32_t header_off = next_off ? 0 : header_size(r->version);\n+\tuint32_t header_off = next_off ? 0 : header_size(t->version);\n \tint32_t block_size = 0;\n \n-\tif (next_off >= r->size)\n+\tif (next_off >= t->size)\n \t\treturn 1;\n \n-\terr = reader_get_block(r, &block, next_off, guess_block_size);\n+\terr = table_get_block(t, &block, next_off, guess_block_size);\n \tif (err < 0)\n \t\tgoto done;\n \n \tblock_size = extract_block_size(block.data, &block_typ, next_off,\n-\t\t\t\t\tr->version);\n+\t\t\t\t\tt->version);\n \tif (block_size < 0) {\n \t\terr = block_size;\n \t\tgoto done;\n@@ -250,14 +250,14 @@ int reader_init_block_reader(struct reftable_reader *r, struct block_reader *br,\n \n \tif (block_size > guess_block_size) {\n \t\treftable_block_done(&block);\n-\t\terr = reader_get_block(r, &block, next_off, block_size);\n+\t\terr = table_get_block(t, &block, next_off, block_size);\n \t\tif (err < 0) {\n \t\t\tgoto done;\n \t\t}\n \t}\n \n-\terr = block_reader_init(br, &block, header_off, r->block_size,\n-\t\t\t\thash_size(r->hash_id));\n+\terr = block_reader_init(br, &block, header_off, t->block_size,\n+\t\t\t\thash_size(t->hash_id));\n done:\n \treftable_block_done(&block);\n \n@@ -268,7 +268,7 @@ static void table_iter_close(struct table_iter *ti)\n {\n \ttable_iter_block_done(ti);\n \tblock_iter_close(&ti->bi);\n-\treftable_reader_decref(ti->r);\n+\treftable_table_decref(ti->table);\n }\n \n static int table_iter_next_block(struct table_iter *ti)\n@@ -276,7 +276,7 @@ static int table_iter_next_block(struct table_iter *ti)\n \tuint64_t next_block_off = ti->block_off + ti->br.full_block_size;\n \tint err;\n \n-\terr = reader_init_block_reader(ti->r, &ti->br, next_block_off, ti->typ);\n+\terr = table_init_block_reader(ti->table, &ti->br, next_block_off, ti->typ);\n \tif (err > 0)\n \t\tti->is_finished = 1;\n \tif (err)\n@@ -326,7 +326,7 @@ static int table_iter_seek_to(struct table_iter *ti, uint64_t off, uint8_t typ)\n {\n \tint err;\n \n-\terr = reader_init_block_reader(ti->r, &ti->br, off, typ);\n+\terr = table_init_block_reader(ti->table, &ti->br, off, typ);\n \tif (err != 0)\n \t\treturn err;\n \n@@ -339,7 +339,7 @@ static int table_iter_seek_to(struct table_iter *ti, uint64_t off, uint8_t typ)\n \n static int table_iter_seek_start(struct table_iter *ti, uint8_t typ, int index)\n {\n-\tstruct reftable_reader_offsets *offs = reader_offsets_for(ti->r, typ);\n+\tstruct reftable_table_offsets *offs = table_offsets_for(ti->table, typ);\n \tuint64_t off = offs->offset;\n \tif (index) {\n \t\toff = offs->index_offset;\n@@ -518,7 +518,7 @@ static int table_iter_seek(struct table_iter *ti,\n \t\t\t   struct reftable_record *want)\n {\n \tuint8_t typ = reftable_record_type(want);\n-\tstruct reftable_reader_offsets *offs = reader_offsets_for(ti->r, typ);\n+\tstruct reftable_table_offsets *offs = table_offsets_for(ti->table, typ);\n \tint err;\n \n \terr = table_iter_seek_start(ti, reftable_record_type(want),\n@@ -566,11 +566,11 @@ static void iterator_from_table_iter(struct reftable_iterator *it,\n \tit->ops = &table_iter_vtable;\n }\n \n-int reader_init_iter(struct reftable_reader *r,\n-\t\t     struct reftable_iterator *it,\n-\t\t     uint8_t typ)\n+int table_init_iter(struct reftable_table *t,\n+\t\t    struct reftable_iterator *it,\n+\t\t    uint8_t typ)\n {\n-\tstruct reftable_reader_offsets *offs = reader_offsets_for(r, typ);\n+\tstruct reftable_table_offsets *offs = table_offsets_for(t, typ);\n \n \tif (offs->is_present) {\n \t\tstruct table_iter *ti;\n@@ -578,7 +578,7 @@ int reader_init_iter(struct reftable_reader *r,\n \t\tif (!ti)\n \t\t\treturn REFTABLE_OUT_OF_MEMORY_ERROR;\n \n-\t\ttable_iter_init(ti, r);\n+\t\ttable_iter_init(ti, t);\n \t\titerator_from_table_iter(it, ti);\n \t} else {\n \t\titerator_set_empty(it);\n@@ -587,31 +587,31 @@ int reader_init_iter(struct reftable_reader *r,\n \treturn 0;\n }\n \n-int reftable_reader_init_ref_iterator(struct reftable_reader *r,\n-\t\t\t\t      struct reftable_iterator *it)\n+int reftable_table_init_ref_iterator(struct reftable_table *t,\n+\t\t\t\t     struct reftable_iterator *it)\n {\n-\treturn reader_init_iter(r, it, BLOCK_TYPE_REF);\n+\treturn table_init_iter(t, it, BLOCK_TYPE_REF);\n }\n \n-int reftable_reader_init_log_iterator(struct reftable_reader *r,\n-\t\t\t\t      struct reftable_iterator *it)\n+int reftable_table_init_log_iterator(struct reftable_table *t,\n+\t\t\t\t     struct reftable_iterator *it)\n {\n-\treturn reader_init_iter(r, it, BLOCK_TYPE_LOG);\n+\treturn table_init_iter(t, it, BLOCK_TYPE_LOG);\n }\n \n-int reftable_reader_new(struct reftable_reader **out,\n-\t\t\tstruct reftable_block_source *source, char const *name)\n+int reftable_table_new(struct reftable_table **out,\n+\t\t       struct reftable_block_source *source, char const *name)\n {\n \tstruct reftable_block footer = { 0 };\n \tstruct reftable_block header = { 0 };\n-\tstruct reftable_reader *r;\n+\tstruct reftable_table *t;\n \tuint64_t file_size = block_source_size(source);\n \tuint32_t read_size;\n \tssize_t bytes_read;\n \tint err;\n \n-\tREFTABLE_CALLOC_ARRAY(r, 1);\n-\tif (!r) {\n+\tREFTABLE_CALLOC_ARRAY(t, 1);\n+\tif (!t) {\n \t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\tgoto done;\n \t}\n@@ -636,72 +636,72 @@ int reftable_reader_new(struct reftable_reader **out,\n \t\terr = REFTABLE_FORMAT_ERROR;\n \t\tgoto done;\n \t}\n-\tr->version = header.data[4];\n-\tif (r->version != 1 && r->version != 2) {\n+\tt->version = header.data[4];\n+\tif (t->version != 1 && t->version != 2) {\n \t\terr = REFTABLE_FORMAT_ERROR;\n \t\tgoto done;\n \t}\n \n-\tr->size = file_size - footer_size(r->version);\n-\tr->source = *source;\n-\tr->name = reftable_strdup(name);\n-\tif (!r->name) {\n+\tt->size = file_size - footer_size(t->version);\n+\tt->source = *source;\n+\tt->name = reftable_strdup(name);\n+\tif (!t->name) {\n \t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\tgoto done;\n \t}\n-\tr->hash_id = 0;\n-\tr->refcount = 1;\n+\tt->hash_id = 0;\n+\tt->refcount = 1;\n \n-\tbytes_read = block_source_read_block(source, &footer, r->size,\n-\t\t\t\t\t     footer_size(r->version));\n-\tif (bytes_read < 0 || (size_t)bytes_read != footer_size(r->version)) {\n+\tbytes_read = block_source_read_block(source, &footer, t->size,\n+\t\t\t\t\t     footer_size(t->version));\n+\tif (bytes_read < 0 || (size_t)bytes_read != footer_size(t->version)) {\n \t\terr = REFTABLE_IO_ERROR;\n \t\tgoto done;\n \t}\n \n-\terr = parse_footer(r, footer.data, header.data);\n+\terr = parse_footer(t, footer.data, header.data);\n \tif (err)\n \t\tgoto done;\n \n-\t*out = r;\n+\t*out = t;\n \n done:\n \treftable_block_done(&footer);\n \treftable_block_done(&header);\n \tif (err) {\n-\t\tif (r)\n-\t\t\treftable_free(r->name);\n-\t\treftable_free(r);\n+\t\tif (t)\n+\t\t\treftable_free(t->name);\n+\t\treftable_free(t);\n \t\tblock_source_close(source);\n \t}\n \treturn err;\n }\n \n-void reftable_reader_incref(struct reftable_reader *r)\n+void reftable_table_incref(struct reftable_table *t)\n {\n-\tr->refcount++;\n+\tt->refcount++;\n }\n \n-void reftable_reader_decref(struct reftable_reader *r)\n+void reftable_table_decref(struct reftable_table *t)\n {\n-\tif (!r)\n+\tif (!t)\n \t\treturn;\n-\tif (--r->refcount)\n+\tif (--t->refcount)\n \t\treturn;\n-\tblock_source_close(&r->source);\n-\tREFTABLE_FREE_AND_NULL(r->name);\n-\treftable_free(r);\n+\tblock_source_close(&t->source);\n+\tREFTABLE_FREE_AND_NULL(t->name);\n+\treftable_free(t);\n }\n \n-static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n-\t\t\t\t\t    struct reftable_iterator *it,\n-\t\t\t\t\t    uint8_t *oid)\n+static int reftable_table_refs_for_indexed(struct reftable_table *t,\n+\t\t\t\t\t   struct reftable_iterator *it,\n+\t\t\t\t\t   uint8_t *oid)\n {\n \tstruct reftable_record want = {\n \t\t.type = BLOCK_TYPE_OBJ,\n \t\t.u.obj = {\n \t\t\t.hash_prefix = oid,\n-\t\t\t.hash_prefix_len = r->object_id_len,\n+\t\t\t.hash_prefix_len = t->object_id_len,\n \t\t},\n \t};\n \tstruct reftable_iterator oit = { NULL };\n@@ -713,7 +713,7 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \tstruct indexed_table_ref_iter *itr = NULL;\n \n \t/* Look through the reverse index. */\n-\terr = reader_init_iter(r, &oit, BLOCK_TYPE_OBJ);\n+\terr = table_init_iter(t, &oit, BLOCK_TYPE_OBJ);\n \tif (err < 0)\n \t\tgoto done;\n \n@@ -727,14 +727,14 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \t\tgoto done;\n \n \tif (err > 0 || memcmp(want.u.obj.hash_prefix, got.u.obj.hash_prefix,\n-\t\t\t      r->object_id_len)) {\n+\t\t\t      t->object_id_len)) {\n \t\t/* didn't find it; return empty iterator */\n \t\titerator_set_empty(it);\n \t\terr = 0;\n \t\tgoto done;\n \t}\n \n-\terr = indexed_table_ref_iter_new(&itr, r, oid, hash_size(r->hash_id),\n+\terr = indexed_table_ref_iter_new(&itr, t, oid, hash_size(t->hash_id),\n \t\t\t\t\t got.u.obj.offsets,\n \t\t\t\t\t got.u.obj.offset_len);\n \tif (err < 0)\n@@ -748,14 +748,14 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \treturn err;\n }\n \n-static int reftable_reader_refs_for_unindexed(struct reftable_reader *r,\n-\t\t\t\t\t      struct reftable_iterator *it,\n-\t\t\t\t\t      uint8_t *oid)\n+static int reftable_table_refs_for_unindexed(struct reftable_table *t,\n+\t\t\t\t\t     struct reftable_iterator *it,\n+\t\t\t\t\t     uint8_t *oid)\n {\n \tstruct table_iter *ti;\n \tstruct filtering_ref_iterator *filter = NULL;\n \tstruct filtering_ref_iterator empty = FILTERING_REF_ITERATOR_INIT;\n-\tuint32_t oid_len = hash_size(r->hash_id);\n+\tuint32_t oid_len = hash_size(t->hash_id);\n \tint err;\n \n \tREFTABLE_ALLOC_ARRAY(ti, 1);\n@@ -764,7 +764,7 @@ static int reftable_reader_refs_for_unindexed(struct reftable_reader *r,\n \t\tgoto out;\n \t}\n \n-\ttable_iter_init(ti, r);\n+\ttable_iter_init(ti, t);\n \terr = table_iter_seek_start(ti, BLOCK_TYPE_REF, 0);\n \tif (err < 0)\n \t\tgoto out;\n@@ -795,25 +795,25 @@ static int reftable_reader_refs_for_unindexed(struct reftable_reader *r,\n \treturn err;\n }\n \n-int reftable_reader_refs_for(struct reftable_reader *r,\n-\t\t\t     struct reftable_iterator *it, uint8_t *oid)\n+int reftable_table_refs_for(struct reftable_table *t,\n+\t\t\t    struct reftable_iterator *it, uint8_t *oid)\n {\n-\tif (r->obj_offsets.is_present)\n-\t\treturn reftable_reader_refs_for_indexed(r, it, oid);\n-\treturn reftable_reader_refs_for_unindexed(r, it, oid);\n+\tif (t->obj_offsets.is_present)\n+\t\treturn reftable_table_refs_for_indexed(t, it, oid);\n+\treturn reftable_table_refs_for_unindexed(t, it, oid);\n }\n \n-uint64_t reftable_reader_max_update_index(struct reftable_reader *r)\n+uint64_t reftable_table_max_update_index(struct reftable_table *t)\n {\n-\treturn r->max_update_index;\n+\treturn t->max_update_index;\n }\n \n-uint64_t reftable_reader_min_update_index(struct reftable_reader *r)\n+uint64_t reftable_table_min_update_index(struct reftable_table *t)\n {\n-\treturn r->min_update_index;\n+\treturn t->min_update_index;\n }\n \n-int reftable_reader_print_blocks(const char *tablename)\n+int reftable_table_print_blocks(const char *tablename)\n {\n \tstruct {\n \t\tconst char *name;\n@@ -833,7 +833,7 @@ int reftable_reader_print_blocks(const char *tablename)\n \t\t},\n \t};\n \tstruct reftable_block_source src = { 0 };\n-\tstruct reftable_reader *r = NULL;\n+\tstruct reftable_table *table = NULL;\n \tstruct table_iter ti = { 0 };\n \tsize_t i;\n \tint err;\n@@ -842,14 +842,14 @@ int reftable_reader_print_blocks(const char *tablename)\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = reftable_reader_new(&r, &src, tablename);\n+\terr = reftable_table_new(&table, &src, tablename);\n \tif (err < 0)\n \t\tgoto done;\n \n-\ttable_iter_init(&ti, r);\n+\ttable_iter_init(&ti, table);\n \n \tprintf(\"header:\\n\");\n-\tprintf(\"  block_size: %d\\n\", r->block_size);\n+\tprintf(\"  block_size: %d\\n\", table->block_size);\n \n \tfor (i = 0; i < sizeof(sections) / sizeof(*sections); i++) {\n \t\terr = table_iter_seek_start(&ti, sections[i].type, 0);\n@@ -873,7 +873,7 @@ int reftable_reader_print_blocks(const char *tablename)\n \t}\n \n done:\n-\treftable_reader_decref(r);\n+\treftable_table_decref(table);\n \ttable_iter_close(&ti);\n \treturn err;\n }\ndiff --git a/reftable/reader.h b/reftable/table.h\nsimilarity index 65%\nrename from reftable/reader.h\nrename to reftable/table.h\nindex c9dccbd6c97..9cd8f80a207 100644\n--- a/reftable/reader.h\n+++ b/reftable/table.h\n@@ -6,13 +6,13 @@\n  * https://developers.google.com/open-source/licenses/bsd\n  */\n \n-#ifndef READER_H\n-#define READER_H\n+#ifndef TABLE_H\n+#define TABLE_H\n \n #include \"block.h\"\n #include \"record.h\"\n #include \"reftable-iterator.h\"\n-#include \"reftable-reader.h\"\n+#include \"reftable-table.h\"\n \n uint64_t block_source_size(struct reftable_block_source *source);\n \n@@ -22,14 +22,14 @@ ssize_t block_source_read_block(struct reftable_block_source *source,\n void block_source_close(struct reftable_block_source *source);\n \n /* metadata for a block type */\n-struct reftable_reader_offsets {\n+struct reftable_table_offsets {\n \tint is_present;\n \tuint64_t offset;\n \tuint64_t index_offset;\n };\n \n /* The state for reading a reftable file. */\n-struct reftable_reader {\n+struct reftable_table {\n \t/* for convenience, associate a name with the instance. */\n \tchar *name;\n \tstruct reftable_block_source source;\n@@ -47,21 +47,21 @@ struct reftable_reader {\n \tint object_id_len;\n \tint version;\n \n-\tstruct reftable_reader_offsets ref_offsets;\n-\tstruct reftable_reader_offsets obj_offsets;\n-\tstruct reftable_reader_offsets log_offsets;\n+\tstruct reftable_table_offsets ref_offsets;\n+\tstruct reftable_table_offsets obj_offsets;\n+\tstruct reftable_table_offsets log_offsets;\n \n \tuint64_t refcount;\n };\n \n-const char *reader_name(struct reftable_reader *r);\n+const char *reftable_table_name(struct reftable_table *t);\n \n-int reader_init_iter(struct reftable_reader *r,\n-\t\t     struct reftable_iterator *it,\n-\t\t     uint8_t typ);\n+int table_init_iter(struct reftable_table *t,\n+\t\t    struct reftable_iterator *it,\n+\t\t    uint8_t typ);\n \n-/* initialize a block reader to read from `r` */\n-int reader_init_block_reader(struct reftable_reader *r, struct block_reader *br,\n-\t\t\t     uint64_t next_off, uint8_t want_typ);\n+/* initialize a block reader to read from `t` */\n+int table_init_block_reader(struct reftable_table *t, struct block_reader *br,\n+\t\t\t    uint64_t next_off, uint8_t want_typ);\n \n #endif\ndiff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c\nindex 3c72ed985b3..f8f1956f4f3 100644\n--- a/t/helper/test-reftable.c\n+++ b/t/helper/test-reftable.c\n@@ -4,8 +4,8 @@\n #include \"reftable/system.h\"\n #include \"reftable/reftable-error.h\"\n #include \"reftable/reftable-merged.h\"\n-#include \"reftable/reftable-reader.h\"\n #include \"reftable/reftable-stack.h\"\n+#include \"reftable/reftable-table.h\"\n #include \"test-tool.h\"\n \n static void print_help(void)\n@@ -126,19 +126,19 @@ static int dump_reftable(const char *tablename)\n {\n \tstruct reftable_block_source src = { 0 };\n \tstruct reftable_merged_table *mt = NULL;\n-\tstruct reftable_reader *r = NULL;\n+\tstruct reftable_table *table = NULL;\n \tint err;\n \n \terr = reftable_block_source_from_file(&src, tablename);\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = reftable_reader_new(&r, &src, tablename);\n+\terr = reftable_table_new(&table, &src, tablename);\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = reftable_merged_table_new(&mt, &r, 1,\n-\t\t\t\t\treftable_reader_hash_id(r));\n+\terr = reftable_merged_table_new(&mt, &table, 1,\n+\t\t\t\t\treftable_table_hash_id(table));\n \tif (err < 0)\n \t\tgoto done;\n \n@@ -146,7 +146,7 @@ static int dump_reftable(const char *tablename)\n \n done:\n \treftable_merged_table_free(mt);\n-\treftable_reader_decref(r);\n+\treftable_table_decref(table);\n \treturn err;\n }\n \n@@ -184,7 +184,7 @@ int cmd__dump_reftable(int argc, const char **argv)\n \targ = argv[1];\n \n \tif (opt_dump_blocks) {\n-\t\terr = reftable_reader_print_blocks(arg);\n+\t\terr = reftable_table_print_blocks(arg);\n \t} else if (opt_dump_table) {\n \t\terr = dump_reftable(arg);\n \t} else if (opt_dump_stack) {\ndiff --git a/t/meson.build b/t/meson.build\nindex a59da26be3f..ded928d9e6d 100644\n--- a/t/meson.build\n+++ b/t/meson.build\n@@ -56,10 +56,10 @@ unit_test_programs = [\n   'unit-tests/t-reftable-block.c',\n   'unit-tests/t-reftable-merged.c',\n   'unit-tests/t-reftable-pq.c',\n-  'unit-tests/t-reftable-reader.c',\n   'unit-tests/t-reftable-readwrite.c',\n   'unit-tests/t-reftable-record.c',\n   'unit-tests/t-reftable-stack.c',\n+  'unit-tests/t-reftable-table.c',\n   'unit-tests/t-trailer.c',\n   'unit-tests/t-urlmatch-normalization.c',\n ]\ndiff --git a/t/unit-tests/t-reftable-merged.c b/t/unit-tests/t-reftable-merged.c\nindex 60836f80d60..fed6beb85c0 100644\n--- a/t/unit-tests/t-reftable-merged.c\n+++ b/t/unit-tests/t-reftable-merged.c\n@@ -11,7 +11,7 @@ license that can be found in the LICENSE file or at\n #include \"reftable/blocksource.h\"\n #include \"reftable/constants.h\"\n #include \"reftable/merged.h\"\n-#include \"reftable/reader.h\"\n+#include \"reftable/table.h\"\n #include \"reftable/reftable-error.h\"\n #include \"reftable/reftable-merged.h\"\n #include \"reftable/reftable-writer.h\"\n@@ -19,7 +19,7 @@ license that can be found in the LICENSE file or at\n static struct reftable_merged_table *\n merged_table_from_records(struct reftable_ref_record **refs,\n \t\t\t  struct reftable_block_source **source,\n-\t\t\t  struct reftable_reader ***readers, const size_t *sizes,\n+\t\t\t  struct reftable_table ***tables, const size_t *sizes,\n \t\t\t  struct reftable_buf *buf, const size_t n)\n {\n \tstruct reftable_merged_table *mt = NULL;\n@@ -28,8 +28,8 @@ merged_table_from_records(struct reftable_ref_record **refs,\n \t};\n \tint err;\n \n-\tREFTABLE_CALLOC_ARRAY(*readers, n);\n-\tcheck(*readers != NULL);\n+\tREFTABLE_CALLOC_ARRAY(*tables, n);\n+\tcheck(*tables != NULL);\n \tREFTABLE_CALLOC_ARRAY(*source, n);\n \tcheck(*source != NULL);\n \n@@ -37,21 +37,21 @@ merged_table_from_records(struct reftable_ref_record **refs,\n \t\tt_reftable_write_to_buf(&buf[i], refs[i], sizes[i], NULL, 0, &opts);\n \t\tblock_source_from_buf(&(*source)[i], &buf[i]);\n \n-\t\terr = reftable_reader_new(&(*readers)[i], &(*source)[i],\n-\t\t\t\t\t  \"name\");\n+\t\terr = reftable_table_new(&(*tables)[i], &(*source)[i],\n+\t\t\t\t\t \"name\");\n \t\tcheck(!err);\n \t}\n \n-\terr = reftable_merged_table_new(&mt, *readers, n, REFTABLE_HASH_SHA1);\n+\terr = reftable_merged_table_new(&mt, *tables, n, REFTABLE_HASH_SHA1);\n \tcheck(!err);\n \treturn mt;\n }\n \n-static void readers_destroy(struct reftable_reader **readers, const size_t n)\n+static void tables_destroy(struct reftable_table **tables, const size_t n)\n {\n \tfor (size_t i = 0; i < n; i++)\n-\t\treftable_reader_decref(readers[i]);\n-\treftable_free(readers);\n+\t\treftable_table_decref(tables[i]);\n+\treftable_free(tables);\n }\n \n static void t_merged_single_record(void)\n@@ -77,9 +77,9 @@ static void t_merged_single_record(void)\n \tsize_t sizes[] = { ARRAY_SIZE(r1), ARRAY_SIZE(r2), ARRAY_SIZE(r3) };\n \tstruct reftable_buf bufs[3] = { REFTABLE_BUF_INIT, REFTABLE_BUF_INIT, REFTABLE_BUF_INIT };\n \tstruct reftable_block_source *bs = NULL;\n-\tstruct reftable_reader **readers = NULL;\n+\tstruct reftable_table **tables = NULL;\n \tstruct reftable_merged_table *mt =\n-\t\tmerged_table_from_records(refs, &bs, &readers, sizes, bufs, 3);\n+\t\tmerged_table_from_records(refs, &bs, &tables, sizes, bufs, 3);\n \tstruct reftable_ref_record ref = { 0 };\n \tstruct reftable_iterator it = { 0 };\n \tint err;\n@@ -94,7 +94,7 @@ static void t_merged_single_record(void)\n \tcheck(reftable_ref_record_equal(&r2[0], &ref, REFTABLE_HASH_SIZE_SHA1));\n \treftable_ref_record_release(&ref);\n \treftable_iterator_destroy(&it);\n-\treaders_destroy(readers, 3);\n+\ttables_destroy(tables, 3);\n \treftable_merged_table_free(mt);\n \tfor (size_t i = 0; i < ARRAY_SIZE(bufs); i++)\n \t\treftable_buf_release(&bufs[i]);\n@@ -154,9 +154,9 @@ static void t_merged_refs(void)\n \tsize_t sizes[3] = { ARRAY_SIZE(r1), ARRAY_SIZE(r2), ARRAY_SIZE(r3) };\n \tstruct reftable_buf bufs[3] = { REFTABLE_BUF_INIT, REFTABLE_BUF_INIT, REFTABLE_BUF_INIT };\n \tstruct reftable_block_source *bs = NULL;\n-\tstruct reftable_reader **readers = NULL;\n+\tstruct reftable_table **tables = NULL;\n \tstruct reftable_merged_table *mt =\n-\t\tmerged_table_from_records(refs, &bs, &readers, sizes, bufs, 3);\n+\t\tmerged_table_from_records(refs, &bs, &tables, sizes, bufs, 3);\n \tstruct reftable_iterator it = { 0 };\n \tint err;\n \tstruct reftable_ref_record *out = NULL;\n@@ -193,7 +193,7 @@ static void t_merged_refs(void)\n \n \tfor (i = 0; i < 3; i++)\n \t\treftable_buf_release(&bufs[i]);\n-\treaders_destroy(readers, 3);\n+\ttables_destroy(tables, 3);\n \treftable_merged_table_free(mt);\n \treftable_free(bs);\n }\n@@ -238,12 +238,12 @@ static void t_merged_seek_multiple_times(void)\n \t\tREFTABLE_BUF_INIT, REFTABLE_BUF_INIT,\n \t};\n \tstruct reftable_block_source *sources = NULL;\n-\tstruct reftable_reader **readers = NULL;\n+\tstruct reftable_table **tables = NULL;\n \tstruct reftable_ref_record rec = { 0 };\n \tstruct reftable_iterator it = { 0 };\n \tstruct reftable_merged_table *mt;\n \n-\tmt = merged_table_from_records(refs, &sources, &readers, sizes, bufs, 2);\n+\tmt = merged_table_from_records(refs, &sources, &tables, sizes, bufs, 2);\n \tmerged_table_init_iter(mt, &it, BLOCK_TYPE_REF);\n \n \tfor (size_t i = 0; i < 5; i++) {\n@@ -266,7 +266,7 @@ static void t_merged_seek_multiple_times(void)\n \n \tfor (size_t i = 0; i < ARRAY_SIZE(bufs); i++)\n \t\treftable_buf_release(&bufs[i]);\n-\treaders_destroy(readers, ARRAY_SIZE(refs));\n+\ttables_destroy(tables, ARRAY_SIZE(refs));\n \treftable_ref_record_release(&rec);\n \treftable_iterator_destroy(&it);\n \treftable_merged_table_free(mt);\n@@ -313,13 +313,13 @@ static void t_merged_seek_multiple_times_without_draining(void)\n \t\tREFTABLE_BUF_INIT, REFTABLE_BUF_INIT,\n \t};\n \tstruct reftable_block_source *sources = NULL;\n-\tstruct reftable_reader **readers = NULL;\n+\tstruct reftable_table **tables = NULL;\n \tstruct reftable_ref_record rec = { 0 };\n \tstruct reftable_iterator it = { 0 };\n \tstruct reftable_merged_table *mt;\n \tint err;\n \n-\tmt = merged_table_from_records(refs, &sources, &readers, sizes, bufs, 2);\n+\tmt = merged_table_from_records(refs, &sources, &tables, sizes, bufs, 2);\n \tmerged_table_init_iter(mt, &it, BLOCK_TYPE_REF);\n \n \terr = reftable_iterator_seek_ref(&it, \"b\");\n@@ -338,7 +338,7 @@ static void t_merged_seek_multiple_times_without_draining(void)\n \n \tfor (size_t i = 0; i < ARRAY_SIZE(bufs); i++)\n \t\treftable_buf_release(&bufs[i]);\n-\treaders_destroy(readers, ARRAY_SIZE(refs));\n+\ttables_destroy(tables, ARRAY_SIZE(refs));\n \treftable_ref_record_release(&rec);\n \treftable_iterator_destroy(&it);\n \treftable_merged_table_free(mt);\n@@ -348,7 +348,7 @@ static void t_merged_seek_multiple_times_without_draining(void)\n static struct reftable_merged_table *\n merged_table_from_log_records(struct reftable_log_record **logs,\n \t\t\t      struct reftable_block_source **source,\n-\t\t\t      struct reftable_reader ***readers, const size_t *sizes,\n+\t\t\t      struct reftable_table ***tables, const size_t *sizes,\n \t\t\t      struct reftable_buf *buf, const size_t n)\n {\n \tstruct reftable_merged_table *mt = NULL;\n@@ -358,8 +358,8 @@ merged_table_from_log_records(struct reftable_log_record **logs,\n \t};\n \tint err;\n \n-\tREFTABLE_CALLOC_ARRAY(*readers, n);\n-\tcheck(*readers != NULL);\n+\tREFTABLE_CALLOC_ARRAY(*tables, n);\n+\tcheck(*tables != NULL);\n \tREFTABLE_CALLOC_ARRAY(*source, n);\n \tcheck(*source != NULL);\n \n@@ -367,12 +367,12 @@ merged_table_from_log_records(struct reftable_log_record **logs,\n \t\tt_reftable_write_to_buf(&buf[i], NULL, 0, logs[i], sizes[i], &opts);\n \t\tblock_source_from_buf(&(*source)[i], &buf[i]);\n \n-\t\terr = reftable_reader_new(&(*readers)[i], &(*source)[i],\n-\t\t\t\t\t  \"name\");\n+\t\terr = reftable_table_new(&(*tables)[i], &(*source)[i],\n+\t\t\t\t\t \"name\");\n \t\tcheck(!err);\n \t}\n \n-\terr = reftable_merged_table_new(&mt, *readers, n, REFTABLE_HASH_SHA1);\n+\terr = reftable_merged_table_new(&mt, *tables, n, REFTABLE_HASH_SHA1);\n \tcheck(!err);\n \treturn mt;\n }\n@@ -435,9 +435,9 @@ static void t_merged_logs(void)\n \tsize_t sizes[3] = { ARRAY_SIZE(r1), ARRAY_SIZE(r2), ARRAY_SIZE(r3) };\n \tstruct reftable_buf bufs[3] = { REFTABLE_BUF_INIT, REFTABLE_BUF_INIT, REFTABLE_BUF_INIT };\n \tstruct reftable_block_source *bs = NULL;\n-\tstruct reftable_reader **readers = NULL;\n+\tstruct reftable_table **tables = NULL;\n \tstruct reftable_merged_table *mt = merged_table_from_log_records(\n-\t\tlogs, &bs, &readers, sizes, bufs, 3);\n+\t\tlogs, &bs, &tables, sizes, bufs, 3);\n \tstruct reftable_iterator it = { 0 };\n \tint err;\n \tstruct reftable_log_record *out = NULL;\n@@ -485,7 +485,7 @@ static void t_merged_logs(void)\n \n \tfor (i = 0; i < 3; i++)\n \t\treftable_buf_release(&bufs[i]);\n-\treaders_destroy(readers, 3);\n+\ttables_destroy(tables, 3);\n \treftable_merged_table_free(mt);\n \treftable_free(bs);\n }\n@@ -502,7 +502,7 @@ static void t_default_write_opts(void)\n \tint err;\n \tstruct reftable_block_source source = { 0 };\n \tuint32_t hash_id;\n-\tstruct reftable_reader *rd = NULL;\n+\tstruct reftable_table *table = NULL;\n \tstruct reftable_merged_table *merged = NULL;\n \n \treftable_writer_set_limits(w, 1, 1);\n@@ -516,18 +516,18 @@ static void t_default_write_opts(void)\n \n \tblock_source_from_buf(&source, &buf);\n \n-\terr = reftable_reader_new(&rd, &source, \"filename\");\n+\terr = reftable_table_new(&table, &source, \"filename\");\n \tcheck(!err);\n \n-\thash_id = reftable_reader_hash_id(rd);\n+\thash_id = reftable_table_hash_id(table);\n \tcheck_int(hash_id, ==, REFTABLE_HASH_SHA1);\n \n-\terr = reftable_merged_table_new(&merged, &rd, 1, REFTABLE_HASH_SHA256);\n+\terr = reftable_merged_table_new(&merged, &table, 1, REFTABLE_HASH_SHA256);\n \tcheck_int(err, ==, REFTABLE_FORMAT_ERROR);\n-\terr = reftable_merged_table_new(&merged, &rd, 1, REFTABLE_HASH_SHA1);\n+\terr = reftable_merged_table_new(&merged, &table, 1, REFTABLE_HASH_SHA1);\n \tcheck(!err);\n \n-\treftable_reader_decref(rd);\n+\treftable_table_decref(table);\n \treftable_merged_table_free(merged);\n \treftable_buf_release(&buf);\n }\ndiff --git a/t/unit-tests/t-reftable-readwrite.c b/t/unit-tests/t-reftable-readwrite.c\nindex c9626831dac..c4c27242ba9 100644\n--- a/t/unit-tests/t-reftable-readwrite.c\n+++ b/t/unit-tests/t-reftable-readwrite.c\n@@ -12,9 +12,9 @@ license that can be found in the LICENSE file or at\n #include \"lib-reftable.h\"\n #include \"reftable/basics.h\"\n #include \"reftable/blocksource.h\"\n-#include \"reftable/reader.h\"\n #include \"reftable/reftable-error.h\"\n #include \"reftable/reftable-writer.h\"\n+#include \"reftable/table.h\"\n #include \"strbuf.h\"\n \n static const int update_index = 5;\n@@ -204,7 +204,7 @@ static void t_log_write_read(void)\n \tstruct reftable_ref_record ref = { 0 };\n \tstruct reftable_log_record log = { 0 };\n \tstruct reftable_iterator it = { 0 };\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tstruct reftable_block_source source = { 0 };\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tstruct reftable_writer *w = t_reftable_strbuf_writer(&buf, &opts);\n@@ -254,10 +254,10 @@ static void t_log_write_read(void)\n \n \tblock_source_from_buf(&source, &buf);\n \n-\terr = reftable_reader_new(&reader, &source, \"file.log\");\n+\terr = reftable_table_new(&table, &source, \"file.log\");\n \tcheck(!err);\n \n-\terr = reftable_reader_init_ref_iterator(reader, &it);\n+\terr = reftable_table_init_ref_iterator(table, &it);\n \tcheck(!err);\n \n \terr = reftable_iterator_seek_ref(&it, names[N - 1]);\n@@ -273,7 +273,7 @@ static void t_log_write_read(void)\n \treftable_iterator_destroy(&it);\n \treftable_ref_record_release(&ref);\n \n-\terr = reftable_reader_init_log_iterator(reader, &it);\n+\terr = reftable_table_init_log_iterator(table, &it);\n \tcheck(!err);\n \terr = reftable_iterator_seek_log(&it, \"\");\n \tcheck(!err);\n@@ -294,7 +294,7 @@ static void t_log_write_read(void)\n \t/* cleanup. */\n \treftable_buf_release(&buf);\n \tfree_names(names);\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n }\n \n static void t_log_zlib_corruption(void)\n@@ -303,7 +303,7 @@ static void t_log_zlib_corruption(void)\n \t\t.block_size = 256,\n \t};\n \tstruct reftable_iterator it = { 0 };\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tstruct reftable_block_source source = { 0 };\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tstruct reftable_writer *w = t_reftable_strbuf_writer(&buf, &opts);\n@@ -345,10 +345,10 @@ static void t_log_zlib_corruption(void)\n \n \tblock_source_from_buf(&source, &buf);\n \n-\terr = reftable_reader_new(&reader, &source, \"file.log\");\n+\terr = reftable_table_new(&table, &source, \"file.log\");\n \tcheck(!err);\n \n-\terr = reftable_reader_init_log_iterator(reader, &it);\n+\terr = reftable_table_init_log_iterator(table, &it);\n \tcheck(!err);\n \terr = reftable_iterator_seek_log(&it, \"refname\");\n \tcheck_int(err, ==, REFTABLE_ZLIB_ERROR);\n@@ -356,7 +356,7 @@ static void t_log_zlib_corruption(void)\n \treftable_iterator_destroy(&it);\n \n \t/* cleanup. */\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n \treftable_buf_release(&buf);\n }\n \n@@ -367,7 +367,7 @@ static void t_table_read_write_sequential(void)\n \tint N = 50;\n \tstruct reftable_iterator it = { 0 };\n \tstruct reftable_block_source source = { 0 };\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tint err = 0;\n \tint j = 0;\n \n@@ -375,10 +375,10 @@ static void t_table_read_write_sequential(void)\n \n \tblock_source_from_buf(&source, &buf);\n \n-\terr = reftable_reader_new(&reader, &source, \"file.ref\");\n+\terr = reftable_table_new(&table, &source, \"file.ref\");\n \tcheck(!err);\n \n-\terr = reftable_reader_init_ref_iterator(reader, &it);\n+\terr = reftable_table_init_ref_iterator(table, &it);\n \tcheck(!err);\n \terr = reftable_iterator_seek_ref(&it, \"\");\n \tcheck(!err);\n@@ -396,7 +396,7 @@ static void t_table_read_write_sequential(void)\n \tcheck_int(j, ==, N);\n \n \treftable_iterator_destroy(&it);\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n \treftable_buf_release(&buf);\n \tfree_names(names);\n }\n@@ -417,7 +417,7 @@ static void t_table_read_api(void)\n \tchar **names;\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tint N = 50;\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tstruct reftable_block_source source = { 0 };\n \tint err;\n \tstruct reftable_log_record log = { 0 };\n@@ -427,10 +427,10 @@ static void t_table_read_api(void)\n \n \tblock_source_from_buf(&source, &buf);\n \n-\terr = reftable_reader_new(&reader, &source, \"file.ref\");\n+\terr = reftable_table_new(&table, &source, \"file.ref\");\n \tcheck(!err);\n \n-\terr = reftable_reader_init_ref_iterator(reader, &it);\n+\terr = reftable_table_init_ref_iterator(table, &it);\n \tcheck(!err);\n \terr = reftable_iterator_seek_ref(&it, names[0]);\n \tcheck(!err);\n@@ -441,7 +441,7 @@ static void t_table_read_api(void)\n \treftable_buf_release(&buf);\n \tfree_names(names);\n \treftable_iterator_destroy(&it);\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n \treftable_buf_release(&buf);\n }\n \n@@ -450,7 +450,7 @@ static void t_table_read_write_seek(int index, enum reftable_hash hash_id)\n \tchar **names;\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tint N = 50;\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tstruct reftable_block_source source = { 0 };\n \tint err;\n \tint i = 0;\n@@ -463,18 +463,18 @@ static void t_table_read_write_seek(int index, enum reftable_hash hash_id)\n \n \tblock_source_from_buf(&source, &buf);\n \n-\terr = reftable_reader_new(&reader, &source, \"file.ref\");\n+\terr = reftable_table_new(&table, &source, \"file.ref\");\n \tcheck(!err);\n-\tcheck_int(hash_id, ==, reftable_reader_hash_id(reader));\n+\tcheck_int(hash_id, ==, reftable_table_hash_id(table));\n \n \tif (!index) {\n-\t\treader->ref_offsets.index_offset = 0;\n+\t\ttable->ref_offsets.index_offset = 0;\n \t} else {\n-\t\tcheck_int(reader->ref_offsets.index_offset, >, 0);\n+\t\tcheck_int(table->ref_offsets.index_offset, >, 0);\n \t}\n \n \tfor (i = 1; i < N; i++) {\n-\t\terr = reftable_reader_init_ref_iterator(reader, &it);\n+\t\terr = reftable_table_init_ref_iterator(table, &it);\n \t\tcheck(!err);\n \t\terr = reftable_iterator_seek_ref(&it, names[i]);\n \t\tcheck(!err);\n@@ -491,7 +491,7 @@ static void t_table_read_write_seek(int index, enum reftable_hash hash_id)\n \tcheck(!reftable_buf_addstr(&pastLast, names[N - 1]));\n \tcheck(!reftable_buf_addstr(&pastLast, \"/\"));\n \n-\terr = reftable_reader_init_ref_iterator(reader, &it);\n+\terr = reftable_table_init_ref_iterator(table, &it);\n \tcheck(!err);\n \terr = reftable_iterator_seek_ref(&it, pastLast.buf);\n \tif (err == 0) {\n@@ -507,7 +507,7 @@ static void t_table_read_write_seek(int index, enum reftable_hash hash_id)\n \n \treftable_buf_release(&buf);\n \tfree_names(names);\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n }\n \n static void t_table_read_write_seek_linear(void)\n@@ -535,7 +535,7 @@ static void t_table_refs_for(int indexed)\n \t\t.block_size = 256,\n \t};\n \tstruct reftable_ref_record ref = { 0 };\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tstruct reftable_block_source source = { 0 };\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tstruct reftable_writer *w = t_reftable_strbuf_writer(&buf, &opts);\n@@ -585,18 +585,18 @@ static void t_table_refs_for(int indexed)\n \n \tblock_source_from_buf(&source, &buf);\n \n-\terr = reftable_reader_new(&reader, &source, \"file.ref\");\n+\terr = reftable_table_new(&table, &source, \"file.ref\");\n \tcheck(!err);\n \tif (!indexed)\n-\t\treader->obj_offsets.is_present = 0;\n+\t\ttable->obj_offsets.is_present = 0;\n \n-\terr = reftable_reader_init_ref_iterator(reader, &it);\n+\terr = reftable_table_init_ref_iterator(table, &it);\n \tcheck(!err);\n \terr = reftable_iterator_seek_ref(&it, \"\");\n \tcheck(!err);\n \treftable_iterator_destroy(&it);\n \n-\terr = reftable_reader_refs_for(reader, &it, want_hash);\n+\terr = reftable_table_refs_for(table, &it, want_hash);\n \tcheck(!err);\n \n \tfor (j = 0; ; j++) {\n@@ -613,7 +613,7 @@ static void t_table_refs_for(int indexed)\n \treftable_buf_release(&buf);\n \tfree_names(want_names);\n \treftable_iterator_destroy(&it);\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n }\n \n static void t_table_refs_for_no_index(void)\n@@ -632,7 +632,7 @@ static void t_write_empty_table(void)\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tstruct reftable_writer *w = t_reftable_strbuf_writer(&buf, &opts);\n \tstruct reftable_block_source source = { 0 };\n-\tstruct reftable_reader *rd = NULL;\n+\tstruct reftable_table *table = NULL;\n \tstruct reftable_ref_record rec = { 0 };\n \tstruct reftable_iterator it = { 0 };\n \tint err;\n@@ -647,10 +647,10 @@ static void t_write_empty_table(void)\n \n \tblock_source_from_buf(&source, &buf);\n \n-\terr = reftable_reader_new(&rd, &source, \"filename\");\n+\terr = reftable_table_new(&table, &source, \"filename\");\n \tcheck(!err);\n \n-\terr = reftable_reader_init_ref_iterator(rd, &it);\n+\terr = reftable_table_init_ref_iterator(table, &it);\n \tcheck(!err);\n \terr = reftable_iterator_seek_ref(&it, \"\");\n \tcheck(!err);\n@@ -659,7 +659,7 @@ static void t_write_empty_table(void)\n \tcheck_int(err, >, 0);\n \n \treftable_iterator_destroy(&it);\n-\treftable_reader_decref(rd);\n+\treftable_table_decref(table);\n \treftable_buf_release(&buf);\n }\n \n@@ -803,7 +803,7 @@ static void t_write_multiple_indices(void)\n \tstruct reftable_iterator it = { 0 };\n \tconst struct reftable_stats *stats;\n \tstruct reftable_writer *writer;\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tchar buf[128];\n \tint err, i;\n \n@@ -852,21 +852,21 @@ static void t_write_multiple_indices(void)\n \tcheck_int(stats->log_stats.index_offset, >, 0);\n \n \tblock_source_from_buf(&source, &writer_buf);\n-\terr = reftable_reader_new(&reader, &source, \"filename\");\n+\terr = reftable_table_new(&table, &source, \"filename\");\n \tcheck(!err);\n \n \t/*\n \t * Seeking the log uses the log index now. In case there is any\n \t * confusion regarding indices we would notice here.\n \t */\n-\terr = reftable_reader_init_log_iterator(reader, &it);\n+\terr = reftable_table_init_log_iterator(table, &it);\n \tcheck(!err);\n \terr = reftable_iterator_seek_log(&it, \"\");\n \tcheck(!err);\n \n \treftable_iterator_destroy(&it);\n \treftable_writer_free(writer);\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n \treftable_buf_release(&writer_buf);\n }\n \n@@ -880,7 +880,7 @@ static void t_write_multi_level_index(void)\n \tstruct reftable_iterator it = { 0 };\n \tconst struct reftable_stats *stats;\n \tstruct reftable_writer *writer;\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tint err;\n \n \twriter = t_reftable_strbuf_writer(&writer_buf, &opts);\n@@ -909,20 +909,20 @@ static void t_write_multi_level_index(void)\n \tcheck_int(stats->ref_stats.max_index_level, ==, 2);\n \n \tblock_source_from_buf(&source, &writer_buf);\n-\terr = reftable_reader_new(&reader, &source, \"filename\");\n+\terr = reftable_table_new(&table, &source, \"filename\");\n \tcheck(!err);\n \n \t/*\n \t * Seeking the last ref should work as expected.\n \t */\n-\terr = reftable_reader_init_ref_iterator(reader, &it);\n+\terr = reftable_table_init_ref_iterator(table, &it);\n \tcheck(!err);\n \terr = reftable_iterator_seek_ref(&it, \"refs/heads/199\");\n \tcheck(!err);\n \n \treftable_iterator_destroy(&it);\n \treftable_writer_free(writer);\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n \treftable_buf_release(&writer_buf);\n \treftable_buf_release(&buf);\n }\n@@ -931,11 +931,11 @@ static void t_corrupt_table_empty(void)\n {\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tstruct reftable_block_source source = { 0 };\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tint err;\n \n \tblock_source_from_buf(&source, &buf);\n-\terr = reftable_reader_new(&reader, &source, \"file.log\");\n+\terr = reftable_table_new(&table, &source, \"file.log\");\n \tcheck_int(err, ==, REFTABLE_FORMAT_ERROR);\n }\n \n@@ -944,12 +944,12 @@ static void t_corrupt_table(void)\n \tuint8_t zeros[1024] = { 0 };\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tstruct reftable_block_source source = { 0 };\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tint err;\n \tcheck(!reftable_buf_add(&buf, zeros, sizeof(zeros)));\n \n \tblock_source_from_buf(&source, &buf);\n-\terr = reftable_reader_new(&reader, &source, \"file.log\");\n+\terr = reftable_table_new(&table, &source, \"file.log\");\n \tcheck_int(err, ==, REFTABLE_FORMAT_ERROR);\n \n \treftable_buf_release(&buf);\ndiff --git a/t/unit-tests/t-reftable-stack.c b/t/unit-tests/t-reftable-stack.c\nindex c3f0059c346..2f49c975194 100644\n--- a/t/unit-tests/t-reftable-stack.c\n+++ b/t/unit-tests/t-reftable-stack.c\n@@ -12,9 +12,9 @@ license that can be found in the LICENSE file or at\n #include \"lib-reftable.h\"\n #include \"dir.h\"\n #include \"reftable/merged.h\"\n-#include \"reftable/reader.h\"\n #include \"reftable/reftable-error.h\"\n #include \"reftable/stack.h\"\n+#include \"reftable/table.h\"\n #include \"strbuf.h\"\n #include \"tempfile.h\"\n #include <dirent.h>\n@@ -176,7 +176,7 @@ static void t_reftable_stack_add_one(void)\n \terr = reftable_stack_read_ref(st, ref.refname, &dest);\n \tcheck(!err);\n \tcheck(reftable_ref_record_equal(&ref, &dest, REFTABLE_HASH_SIZE_SHA1));\n-\tcheck_int(st->readers_len, >, 0);\n+\tcheck_int(st->tables_len, >, 0);\n \n #ifndef GIT_WINDOWS_NATIVE\n \tcheck(!reftable_buf_addstr(&scratch, dir));\n@@ -189,7 +189,7 @@ static void t_reftable_stack_add_one(void)\n \tcheck(!reftable_buf_addstr(&scratch, dir));\n \tcheck(!reftable_buf_addstr(&scratch, \"/\"));\n \t/* do not try at home; not an external API for reftable. */\n-\tcheck(!reftable_buf_addstr(&scratch, st->readers[0]->name));\n+\tcheck(!reftable_buf_addstr(&scratch, st->tables[0]->name));\n \terr = stat(scratch.buf, &stat_result);\n \tcheck(!err);\n \tcheck_int((stat_result.st_mode & 0777), ==, opts.default_permissions);\n@@ -402,9 +402,9 @@ static void t_reftable_stack_transaction_api_performs_auto_compaction(void)\n \t\t * all tables in the stack.\n \t\t */\n \t\tif (i != n)\n-\t\t\tcheck_int(st->merged->readers_len, ==, i + 1);\n+\t\t\tcheck_int(st->merged->tables_len, ==, i + 1);\n \t\telse\n-\t\t\tcheck_int(st->merged->readers_len, ==, 1);\n+\t\t\tcheck_int(st->merged->tables_len, ==, 1);\n \t}\n \n \treftable_stack_destroy(st);\n@@ -430,7 +430,7 @@ static void t_reftable_stack_auto_compaction_fails_gracefully(void)\n \n \terr = reftable_stack_add(st, write_test_ref, &ref);\n \tcheck(!err);\n-\tcheck_int(st->merged->readers_len, ==, 1);\n+\tcheck_int(st->merged->tables_len, ==, 1);\n \tcheck_int(st->stats.attempts, ==, 0);\n \tcheck_int(st->stats.failures, ==, 0);\n \n@@ -441,14 +441,14 @@ static void t_reftable_stack_auto_compaction_fails_gracefully(void)\n \t */\n \tcheck(!reftable_buf_addstr(&table_path, dir));\n \tcheck(!reftable_buf_addstr(&table_path, \"/\"));\n-\tcheck(!reftable_buf_addstr(&table_path, st->readers[0]->name));\n+\tcheck(!reftable_buf_addstr(&table_path, st->tables[0]->name));\n \tcheck(!reftable_buf_addstr(&table_path, \".lock\"));\n \twrite_file_buf(table_path.buf, \"\", 0);\n \n \tref.update_index = 2;\n \terr = reftable_stack_add(st, write_test_ref, &ref);\n \tcheck(!err);\n-\tcheck_int(st->merged->readers_len, ==, 2);\n+\tcheck_int(st->merged->tables_len, ==, 2);\n \tcheck_int(st->stats.attempts, ==, 1);\n \tcheck_int(st->stats.failures, ==, 1);\n \n@@ -592,7 +592,7 @@ static void t_reftable_stack_add(void)\n \tcheck(!reftable_buf_addstr(&path, dir));\n \tcheck(!reftable_buf_addstr(&path, \"/\"));\n \t/* do not try at home; not an external API for reftable. */\n-\tcheck(!reftable_buf_addstr(&path, st->readers[0]->name));\n+\tcheck(!reftable_buf_addstr(&path, st->tables[0]->name));\n \terr = stat(path.buf, &stat_result);\n \tcheck(!err);\n \tcheck_int((stat_result.st_mode & 0777), ==, opts.default_permissions);\n@@ -1026,7 +1026,7 @@ static void t_reftable_stack_auto_compaction(void)\n \n \t\terr = reftable_stack_auto_compact(st);\n \t\tcheck(!err);\n-\t\tcheck(i < 2 || st->merged->readers_len < 2 * fastlogN(i, 2));\n+\t\tcheck(i < 2 || st->merged->tables_len < 2 * fastlogN(i, 2));\n \t}\n \n \tcheck_int(reftable_stack_compaction_stats(st)->entries_written, <,\n@@ -1061,7 +1061,7 @@ static void t_reftable_stack_auto_compaction_factor(void)\n \t\terr = reftable_stack_add(st, &write_test_ref, &ref);\n \t\tcheck(!err);\n \n-\t\tcheck(i < 5 || st->merged->readers_len < 5 * fastlogN(i, 5));\n+\t\tcheck(i < 5 || st->merged->tables_len < 5 * fastlogN(i, 5));\n \t}\n \n \treftable_stack_destroy(st);\n@@ -1082,7 +1082,7 @@ static void t_reftable_stack_auto_compaction_with_locked_tables(void)\n \tcheck(!err);\n \n \twrite_n_ref_tables(st, 5);\n-\tcheck_int(st->merged->readers_len, ==, 5);\n+\tcheck_int(st->merged->tables_len, ==, 5);\n \n \t/*\n \t * Given that all tables we have written should be roughly the same\n@@ -1091,7 +1091,7 @@ static void t_reftable_stack_auto_compaction_with_locked_tables(void)\n \t */\n \tcheck(!reftable_buf_addstr(&buf, dir));\n \tcheck(!reftable_buf_addstr(&buf, \"/\"));\n-\tcheck(!reftable_buf_addstr(&buf, st->readers[2]->name));\n+\tcheck(!reftable_buf_addstr(&buf, st->tables[2]->name));\n \tcheck(!reftable_buf_addstr(&buf, \".lock\"));\n \twrite_file_buf(buf.buf, \"\", 0);\n \n@@ -1104,7 +1104,7 @@ static void t_reftable_stack_auto_compaction_with_locked_tables(void)\n \terr = reftable_stack_auto_compact(st);\n \tcheck(!err);\n \tcheck_int(st->stats.failures, ==, 0);\n-\tcheck_int(st->merged->readers_len, ==, 4);\n+\tcheck_int(st->merged->tables_len, ==, 4);\n \n \treftable_stack_destroy(st);\n \treftable_buf_release(&buf);\n@@ -1149,9 +1149,9 @@ static void t_reftable_stack_add_performs_auto_compaction(void)\n \t\t * all tables in the stack.\n \t\t */\n \t\tif (i != n)\n-\t\t\tcheck_int(st->merged->readers_len, ==, i + 1);\n+\t\t\tcheck_int(st->merged->tables_len, ==, i + 1);\n \t\telse\n-\t\t\tcheck_int(st->merged->readers_len, ==, 1);\n+\t\t\tcheck_int(st->merged->tables_len, ==, 1);\n \t}\n \n \treftable_stack_destroy(st);\n@@ -1172,12 +1172,12 @@ static void t_reftable_stack_compaction_with_locked_tables(void)\n \tcheck(!err);\n \n \twrite_n_ref_tables(st, 3);\n-\tcheck_int(st->merged->readers_len, ==, 3);\n+\tcheck_int(st->merged->tables_len, ==, 3);\n \n \t/* Lock one of the tables that we're about to compact. */\n \tcheck(!reftable_buf_addstr(&buf, dir));\n \tcheck(!reftable_buf_addstr(&buf, \"/\"));\n-\tcheck(!reftable_buf_addstr(&buf, st->readers[1]->name));\n+\tcheck(!reftable_buf_addstr(&buf, st->tables[1]->name));\n \tcheck(!reftable_buf_addstr(&buf, \".lock\"));\n \twrite_file_buf(buf.buf, \"\", 0);\n \n@@ -1188,7 +1188,7 @@ static void t_reftable_stack_compaction_with_locked_tables(void)\n \terr = reftable_stack_compact_all(st, NULL);\n \tcheck_int(err, ==, REFTABLE_LOCK_ERROR);\n \tcheck_int(st->stats.failures, ==, 1);\n-\tcheck_int(st->merged->readers_len, ==, 3);\n+\tcheck_int(st->merged->tables_len, ==, 3);\n \n \treftable_stack_destroy(st);\n \treftable_buf_release(&buf);\n@@ -1222,10 +1222,10 @@ static void t_reftable_stack_compaction_concurrent(void)\n static void unclean_stack_close(struct reftable_stack *st)\n {\n \t/* break abstraction boundary to simulate unclean shutdown. */\n-\tfor (size_t i = 0; i < st->readers_len; i++)\n-\t\treftable_reader_decref(st->readers[i]);\n-\tst->readers_len = 0;\n-\tREFTABLE_FREE_AND_NULL(st->readers);\n+\tfor (size_t i = 0; i < st->tables_len; i++)\n+\t\treftable_table_decref(st->tables[i]);\n+\tst->tables_len = 0;\n+\tREFTABLE_FREE_AND_NULL(st->tables);\n }\n \n static void t_reftable_stack_compaction_concurrent_clean(void)\n@@ -1275,7 +1275,7 @@ static void t_reftable_stack_read_across_reload(void)\n \terr = reftable_new_stack(&st1, dir, &opts);\n \tcheck(!err);\n \twrite_n_ref_tables(st1, 2);\n-\tcheck_int(st1->merged->readers_len, ==, 2);\n+\tcheck_int(st1->merged->tables_len, ==, 2);\n \treftable_stack_init_ref_iterator(st1, &it);\n \terr = reftable_iterator_seek_ref(&it, \"\");\n \tcheck(!err);\n@@ -1283,10 +1283,10 @@ static void t_reftable_stack_read_across_reload(void)\n \t/* Set up a second stack for the same directory and compact it. */\n \terr = reftable_new_stack(&st2, dir, &opts);\n \tcheck(!err);\n-\tcheck_int(st2->merged->readers_len, ==, 2);\n+\tcheck_int(st2->merged->tables_len, ==, 2);\n \terr = reftable_stack_compact_all(st2, NULL);\n \tcheck(!err);\n-\tcheck_int(st2->merged->readers_len, ==, 1);\n+\tcheck_int(st2->merged->tables_len, ==, 1);\n \n \t/*\n \t * Verify that we can continue to use the old iterator even after we\n@@ -1294,7 +1294,7 @@ static void t_reftable_stack_read_across_reload(void)\n \t */\n \terr = reftable_stack_reload(st1);\n \tcheck(!err);\n-\tcheck_int(st1->merged->readers_len, ==, 1);\n+\tcheck_int(st1->merged->tables_len, ==, 1);\n \terr = reftable_iterator_next_ref(&it, &rec);\n \tcheck(!err);\n \tcheck_str(rec.refname, \"refs/heads/branch-0000\");\n@@ -1325,19 +1325,19 @@ static void t_reftable_stack_reload_with_missing_table(void)\n \terr = reftable_new_stack(&st, dir, &opts);\n \tcheck(!err);\n \twrite_n_ref_tables(st, 2);\n-\tcheck_int(st->merged->readers_len, ==, 2);\n+\tcheck_int(st->merged->tables_len, ==, 2);\n \treftable_stack_init_ref_iterator(st, &it);\n \terr = reftable_iterator_seek_ref(&it, \"\");\n \tcheck(!err);\n \n \t/*\n \t * Update the tables.list file with some garbage data, while reusing\n-\t * our old readers. This should trigger a partial reload of the stack,\n-\t * where we try to reuse our old readers.\n+\t * our old tables. This should trigger a partial reload of the stack,\n+\t * where we try to reuse our old tables.\n \t*/\n-\tcheck(!reftable_buf_addstr(&content, st->readers[0]->name));\n+\tcheck(!reftable_buf_addstr(&content, st->tables[0]->name));\n \tcheck(!reftable_buf_addstr(&content, \"\\n\"));\n-\tcheck(!reftable_buf_addstr(&content, st->readers[1]->name));\n+\tcheck(!reftable_buf_addstr(&content, st->tables[1]->name));\n \tcheck(!reftable_buf_addstr(&content, \"\\n\"));\n \tcheck(!reftable_buf_addstr(&content, \"garbage\\n\"));\n \tcheck(!reftable_buf_addstr(&table_path, st->list_file));\n@@ -1348,7 +1348,7 @@ static void t_reftable_stack_reload_with_missing_table(void)\n \n \terr = reftable_stack_reload(st);\n \tcheck_int(err, ==, -4);\n-\tcheck_int(st->merged->readers_len, ==, 2);\n+\tcheck_int(st->merged->tables_len, ==, 2);\n \n \t/*\n \t * Even though the reload has failed, we should be able to continue\ndiff --git a/t/unit-tests/t-reftable-reader.c b/t/unit-tests/t-reftable-table.c\nsimilarity index 78%\nrename from t/unit-tests/t-reftable-reader.c\nrename to t/unit-tests/t-reftable-table.c\nindex 546df6005e4..77c59dbf46d 100644\n--- a/t/unit-tests/t-reftable-reader.c\n+++ b/t/unit-tests/t-reftable-table.c\n@@ -1,9 +1,9 @@\n #include \"test-lib.h\"\n #include \"lib-reftable.h\"\n #include \"reftable/blocksource.h\"\n-#include \"reftable/reader.h\"\n+#include \"reftable/table.h\"\n \n-static int t_reader_seek_once(void)\n+static int t_table_seek_once(void)\n {\n \tstruct reftable_ref_record records[] = {\n \t\t{\n@@ -15,17 +15,17 @@ static int t_reader_seek_once(void)\n \tstruct reftable_block_source source = { 0 };\n \tstruct reftable_ref_record ref = { 0 };\n \tstruct reftable_iterator it = { 0 };\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tint ret;\n \n \tt_reftable_write_to_buf(&buf, records, ARRAY_SIZE(records), NULL, 0, NULL);\n \tblock_source_from_buf(&source, &buf);\n \n-\tret = reftable_reader_new(&reader, &source, \"name\");\n+\tret = reftable_table_new(&table, &source, \"name\");\n \tcheck(!ret);\n \n-\treftable_reader_init_ref_iterator(reader, &it);\n+\treftable_table_init_ref_iterator(table, &it);\n \tret = reftable_iterator_seek_ref(&it, \"\");\n \tcheck(!ret);\n \tret = reftable_iterator_next_ref(&it, &ref);\n@@ -39,12 +39,12 @@ static int t_reader_seek_once(void)\n \n \treftable_ref_record_release(&ref);\n \treftable_iterator_destroy(&it);\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n \treftable_buf_release(&buf);\n \treturn 0;\n }\n \n-static int t_reader_reseek(void)\n+static int t_table_reseek(void)\n {\n \tstruct reftable_ref_record records[] = {\n \t\t{\n@@ -56,17 +56,17 @@ static int t_reader_reseek(void)\n \tstruct reftable_block_source source = { 0 };\n \tstruct reftable_ref_record ref = { 0 };\n \tstruct reftable_iterator it = { 0 };\n-\tstruct reftable_reader *reader;\n+\tstruct reftable_table *table;\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tint ret;\n \n \tt_reftable_write_to_buf(&buf, records, ARRAY_SIZE(records), NULL, 0, NULL);\n \tblock_source_from_buf(&source, &buf);\n \n-\tret = reftable_reader_new(&reader, &source, \"name\");\n+\tret = reftable_table_new(&table, &source, \"name\");\n \tcheck(!ret);\n \n-\treftable_reader_init_ref_iterator(reader, &it);\n+\treftable_table_init_ref_iterator(table, &it);\n \n \tfor (size_t i = 0; i < 5; i++) {\n \t\tret = reftable_iterator_seek_ref(&it, \"\");\n@@ -83,14 +83,14 @@ static int t_reader_reseek(void)\n \n \treftable_ref_record_release(&ref);\n \treftable_iterator_destroy(&it);\n-\treftable_reader_decref(reader);\n+\treftable_table_decref(table);\n \treftable_buf_release(&buf);\n \treturn 0;\n }\n \n int cmd_main(int argc UNUSED, const char *argv[] UNUSED)\n {\n-\tTEST(t_reader_seek_once(), \"reader can seek once\");\n-\tTEST(t_reader_reseek(), \"reader can reseek multiple times\");\n+\tTEST(t_table_seek_once(), \"table can seek once\");\n+\tTEST(t_table_reseek(), \"table can reseek multiple times\");\n \treturn test_done();\n }\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515765","messageId":"20250407-pks-reftable-polishing-v2-4-316c4ff10571@pks.im","threadId":"63222","inReplyTo":"20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im","subject":"[PATCH v2 04/16] reftable/block: simplify how we track restart points","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T13:16:16Z","receivedAt":"2025-04-07T13:16:27Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Restart points record the location of reftable records that do not use\nprefix compression and are used to perform a binary search inside of a\nblock. These restart points are encoded at the end of a block, between\nthe record data and the footer of a table.\n\nThe block structure contains three different variables related to these\nrestart points:\n\n  - The block length contains the length of the reftable block up to the\n    restart points.\n\n  - The restart count contains the number of restart points contained in\n    the block.\n\n  - The restart bytes variable tracks where the restart point data\n    begins.\n\nTracking all three of these variables is unnecessary though as the data\ncan be derived from one another: the block length without restart points\nis the exact same as the offset of the restart count data, which we\nalready track via the `restart_bytes` data.\n\nRefactor the code so that we track the location of restart bytes not as\na pointer, but instead as an offset. This allows us to trivially get rid\nof the `block_len` variable as described above. This avoids having the\nconfusing `block_len` variable and allows us to do less bookkeeping\noverall.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/block.c | 25 ++++++++++++-------------\n reftable/block.h |  8 +++++---\n reftable/table.c |  2 +-\n 3 files changed, 18 insertions(+), 17 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 97740187259..f2567a8f0fd 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -216,10 +216,9 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tuint32_t full_block_size = table_block_size;\n \tuint8_t typ = block->data[header_off];\n \tuint32_t sz = reftable_get_be24(block->data + header_off + 1);\n-\tint err = 0;\n-\tuint16_t restart_count = 0;\n-\tuint32_t restart_start = 0;\n-\tuint8_t *restart_bytes = NULL;\n+\tuint16_t restart_count;\n+\tuint32_t restart_off;\n+\tint err;\n \n \tblock_source_return_block(&br->block);\n \n@@ -300,8 +299,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t}\n \n \trestart_count = reftable_get_be16(block->data + sz - 2);\n-\trestart_start = sz - 2 - 3 * restart_count;\n-\trestart_bytes = block->data + restart_start;\n+\trestart_off = sz - 2 - 3 * restart_count;\n \n \t/* transfer ownership. */\n \tbr->block = *block;\n@@ -309,11 +307,12 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tblock->len = 0;\n \n \tbr->hash_size = hash_size;\n-\tbr->block_len = restart_start;\n+\tbr->restart_off = restart_off;\n \tbr->full_block_size = full_block_size;\n \tbr->header_off = header_off;\n \tbr->restart_count = restart_count;\n-\tbr->restart_bytes = restart_bytes;\n+\n+\terr = 0;\n \n done:\n \treturn err;\n@@ -337,7 +336,7 @@ int block_reader_first_key(const struct block_reader *br, struct reftable_buf *k\n \tint off = br->header_off + 4, n;\n \tstruct string_view in = {\n \t\t.buf = br->block.data + off,\n-\t\t.len = br->block_len - off,\n+\t\t.len = br->restart_off - off,\n \t};\n \tuint8_t extra = 0;\n \n@@ -354,13 +353,13 @@ int block_reader_first_key(const struct block_reader *br, struct reftable_buf *k\n \n static uint32_t block_reader_restart_offset(const struct block_reader *br, size_t idx)\n {\n-\treturn reftable_get_be24(br->restart_bytes + 3 * idx);\n+\treturn reftable_get_be24(br->block.data + br->restart_off + 3 * idx);\n }\n \n void block_iter_seek_start(struct block_iter *it, const struct block_reader *br)\n {\n \tit->block = br->block.data;\n-\tit->block_len = br->block_len;\n+\tit->block_len = br->restart_off;\n \tit->hash_size = br->hash_size;\n \treftable_buf_reset(&it->last_key);\n \tit->next_off = br->header_off + 4;\n@@ -378,7 +377,7 @@ static int restart_needle_less(size_t idx, void *_args)\n \tuint32_t off = block_reader_restart_offset(args->reader, idx);\n \tstruct string_view in = {\n \t\t.buf = args->reader->block.data + off,\n-\t\t.len = args->reader->block_len - off,\n+\t\t.len = args->reader->restart_off - off,\n \t};\n \tuint64_t prefix_len, suffix_len;\n \tuint8_t extra;\n@@ -505,7 +504,7 @@ int block_iter_seek_key(struct block_iter *it, const struct block_reader *br,\n \telse\n \t\tit->next_off = br->header_off + 4;\n \tit->block = br->block.data;\n-\tit->block_len = br->block_len;\n+\tit->block_len = br->restart_off;\n \tit->hash_size = br->hash_size;\n \n \terr = reftable_record_init(&rec, block_reader_type(br));\ndiff --git a/reftable/block.h b/reftable/block.h\nindex 203b07d9a44..b78f322e646 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -79,10 +79,12 @@ struct block_reader {\n \tunsigned char *uncompressed_data;\n \tsize_t uncompressed_cap;\n \n-\t/* size of the data, excluding restart data. */\n-\tuint32_t block_len;\n-\tuint8_t *restart_bytes;\n+\t/*\n+\t * Restart point data. Restart points are located after the block's\n+\t * record data.\n+\t */\n \tuint16_t restart_count;\n+\tuint32_t restart_off;\n \n \t/* size of the data in the file. For log blocks, this is the compressed\n \t * size. */\ndiff --git a/reftable/table.c b/reftable/table.c\nindex d18e17b0d44..ec84545707c 100644\n--- a/reftable/table.c\n+++ b/reftable/table.c\n@@ -838,7 +838,7 @@ int reftable_table_print_blocks(const char *tablename)\n \t\tprintf(\"%s:\\n\", sections[i].name);\n \n \t\twhile (1) {\n-\t\t\tprintf(\"  - length: %u\\n\", ti.br.block_len);\n+\t\t\tprintf(\"  - length: %u\\n\", ti.br.restart_off);\n \t\t\tprintf(\"    restarts: %u\\n\", ti.br.restart_count);\n \n \t\t\terr = table_iter_next_block(&ti);\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515766","messageId":"20250407-pks-reftable-polishing-v2-6-316c4ff10571@pks.im","threadId":"63222","inReplyTo":"20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im","subject":"[PATCH v2 06/16] reftable/block: rename `block` to `block_data`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T13:16:18Z","receivedAt":"2025-04-07T13:16:28Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The `reftable_block` structure associates a byte slice with a block\nsource. As such it only holds the data of a reftable block without\nactually encoding any of the details for how to access that data.\n\nRename the structure to instead be called `reftable_block_data`. Besides\nclarifying that this really only holds data, it also allows us to rename\nthe `reftable_block_reader` to `reftable_block` in the next commit, as\nthis is the structure that actually encapsulates access to the reftable\nblocks.\n\nRename the `struct reftable_block_reader::block` member accordingly.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/block.c                    | 46 ++++++++++++++++++-------------------\n reftable/block.h                    |  2 +-\n reftable/blocksource.c              | 44 +++++++++++++++++------------------\n reftable/blocksource.h              | 10 ++++----\n reftable/iter.c                     |  4 ++--\n reftable/reftable-blocksource.h     | 14 +++++------\n reftable/table.c                    | 16 ++++++-------\n t/unit-tests/t-reftable-readwrite.c | 10 ++++----\n 8 files changed, 73 insertions(+), 73 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 2517108b8ef..dca30b8527f 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -210,16 +210,16 @@ int block_writer_finish(struct block_writer *w)\n }\n \n static int read_block(struct reftable_block_source *source,\n-\t\t      struct reftable_block *dest, uint64_t off,\n+\t\t      struct reftable_block_data *dest, uint64_t off,\n \t\t      uint32_t sz)\n {\n \tsize_t size = block_source_size(source);\n-\tblock_source_return_block(dest);\n+\tblock_source_release_data(dest);\n \tif (off >= size)\n \t\treturn 0;\n \tif (off + sz > size)\n \t\tsz = size - off;\n-\treturn block_source_read_block(source, dest, off, sz);\n+\treturn block_source_read_data(source, dest, off, sz);\n }\n \n int block_reader_init(struct block_reader *br,\n@@ -236,19 +236,19 @@ int block_reader_init(struct block_reader *br,\n \tuint8_t block_type;\n \tint err;\n \n-\terr = read_block(source, &br->block, offset, guess_block_size);\n+\terr = read_block(source, &br->block_data, offset, guess_block_size);\n \tif (err < 0)\n \t\tgoto done;\n \n-\tblock_type = br->block.data[header_size];\n+\tblock_type = br->block_data.data[header_size];\n \tif (!reftable_is_block_type(block_type)) {\n \t\terr = REFTABLE_FORMAT_ERROR;\n \t\tgoto done;\n \t}\n \n-\tblock_size = reftable_get_be24(br->block.data + header_size + 1);\n+\tblock_size = reftable_get_be24(br->block_data.data + header_size + 1);\n \tif (block_size > guess_block_size) {\n-\t\terr = read_block(source, &br->block, offset, block_size);\n+\t\terr = read_block(source, &br->block_data, offset, block_size);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \t}\n@@ -256,7 +256,7 @@ int block_reader_init(struct block_reader *br,\n \tif (block_type == BLOCK_TYPE_LOG) {\n \t\tuint32_t block_header_skip = 4 + header_size;\n \t\tuLong dst_len = block_size - block_header_skip;\n-\t\tuLong src_len = br->block.len - block_header_skip;\n+\t\tuLong src_len = br->block_data.len - block_header_skip;\n \n \t\t/* Log blocks specify the *uncompressed* size in their header. */\n \t\tREFTABLE_ALLOC_GROW_OR_NULL(br->uncompressed_data, block_size,\n@@ -267,7 +267,7 @@ int block_reader_init(struct block_reader *br,\n \t\t}\n \n \t\t/* Copy over the block header verbatim. It's not compressed. */\n-\t\tmemcpy(br->uncompressed_data, br->block.data, block_header_skip);\n+\t\tmemcpy(br->uncompressed_data, br->block_data.data, block_header_skip);\n \n \t\tif (!br->zstream) {\n \t\t\tREFTABLE_CALLOC_ARRAY(br->zstream, 1);\n@@ -285,7 +285,7 @@ int block_reader_init(struct block_reader *br,\n \t\t\tgoto done;\n \t\t}\n \n-\t\tbr->zstream->next_in = br->block.data + block_header_skip;\n+\t\tbr->zstream->next_in = br->block_data.data + block_header_skip;\n \t\tbr->zstream->avail_in = src_len;\n \t\tbr->zstream->next_out = br->uncompressed_data + block_header_skip;\n \t\tbr->zstream->avail_out = dst_len;\n@@ -310,21 +310,21 @@ int block_reader_init(struct block_reader *br,\n \t\t}\n \n \t\t/* We're done with the input data. */\n-\t\tblock_source_return_block(&br->block);\n-\t\tbr->block.data = br->uncompressed_data;\n-\t\tbr->block.len = block_size;\n+\t\tblock_source_release_data(&br->block_data);\n+\t\tbr->block_data.data = br->uncompressed_data;\n+\t\tbr->block_data.len = block_size;\n \t\tfull_block_size = src_len + block_header_skip - br->zstream->avail_in;\n \t} else if (full_block_size == 0) {\n \t\tfull_block_size = block_size;\n-\t} else if (block_size < full_block_size && block_size < br->block.len &&\n-\t\t   br->block.data[block_size] != 0) {\n+\t} else if (block_size < full_block_size && block_size < br->block_data.len &&\n+\t\t   br->block_data.data[block_size] != 0) {\n \t\t/* If the block is smaller than the full block size, it is\n \t\t   padded (data followed by '\\0') or the next block is\n \t\t   unaligned. */\n \t\tfull_block_size = block_size;\n \t}\n \n-\trestart_count = reftable_get_be16(br->block.data + block_size - 2);\n+\trestart_count = reftable_get_be16(br->block_data.data + block_size - 2);\n \trestart_off = block_size - 2 - 3 * restart_count;\n \n \tbr->block_type = block_type;\n@@ -347,20 +347,20 @@ void block_reader_release(struct block_reader *br)\n \tinflateEnd(br->zstream);\n \treftable_free(br->zstream);\n \treftable_free(br->uncompressed_data);\n-\tblock_source_return_block(&br->block);\n+\tblock_source_release_data(&br->block_data);\n \tmemset(br, 0, sizeof(*br));\n }\n \n uint8_t block_reader_type(const struct block_reader *r)\n {\n-\treturn r->block.data[r->header_off];\n+\treturn r->block_data.data[r->header_off];\n }\n \n int block_reader_first_key(const struct block_reader *br, struct reftable_buf *key)\n {\n \tint off = br->header_off + 4, n;\n \tstruct string_view in = {\n-\t\t.buf = br->block.data + off,\n+\t\t.buf = br->block_data.data + off,\n \t\t.len = br->restart_off - off,\n \t};\n \tuint8_t extra = 0;\n@@ -378,12 +378,12 @@ int block_reader_first_key(const struct block_reader *br, struct reftable_buf *k\n \n static uint32_t block_reader_restart_offset(const struct block_reader *br, size_t idx)\n {\n-\treturn reftable_get_be24(br->block.data + br->restart_off + 3 * idx);\n+\treturn reftable_get_be24(br->block_data.data + br->restart_off + 3 * idx);\n }\n \n void block_iter_seek_start(struct block_iter *it, const struct block_reader *br)\n {\n-\tit->block = br->block.data;\n+\tit->block = br->block_data.data;\n \tit->block_len = br->restart_off;\n \tit->hash_size = br->hash_size;\n \treftable_buf_reset(&it->last_key);\n@@ -401,7 +401,7 @@ static int restart_needle_less(size_t idx, void *_args)\n \tstruct restart_needle_less_args *args = _args;\n \tuint32_t off = block_reader_restart_offset(args->reader, idx);\n \tstruct string_view in = {\n-\t\t.buf = args->reader->block.data + off,\n+\t\t.buf = args->reader->block_data.data + off,\n \t\t.len = args->reader->restart_off - off,\n \t};\n \tuint64_t prefix_len, suffix_len;\n@@ -528,7 +528,7 @@ int block_iter_seek_key(struct block_iter *it, const struct block_reader *br,\n \t\tit->next_off = block_reader_restart_offset(br, i - 1);\n \telse\n \t\tit->next_off = br->header_off + 4;\n-\tit->block = br->block.data;\n+\tit->block = br->block_data.data;\n \tit->block_len = br->restart_off;\n \tit->hash_size = br->hash_size;\n \ndiff --git a/reftable/block.h b/reftable/block.h\nindex 6afb1b2952a..948b79e4614 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -71,7 +71,7 @@ struct block_reader {\n \tuint32_t header_off;\n \n \t/* the memory block */\n-\tstruct reftable_block block;\n+\tstruct reftable_block_data block_data;\n \tuint32_t hash_size;\n \n \t/* Uncompressed data for log entries. */\ndiff --git a/reftable/blocksource.c b/reftable/blocksource.c\nindex bc785506fb1..573c81287fe 100644\n--- a/reftable/blocksource.c\n+++ b/reftable/blocksource.c\n@@ -13,15 +13,15 @@\n #include \"reftable-blocksource.h\"\n #include \"reftable-error.h\"\n \n-void block_source_return_block(struct reftable_block *block)\n+void block_source_release_data(struct reftable_block_data *data)\n {\n-\tstruct reftable_block_source source = block->source;\n-\tif (block && source.ops)\n-\t\tsource.ops->return_block(source.arg, block);\n-\tblock->data = NULL;\n-\tblock->len = 0;\n-\tblock->source.ops = NULL;\n-\tblock->source.arg = NULL;\n+\tstruct reftable_block_source source = data->source;\n+\tif (data && source.ops)\n+\t\tsource.ops->release_data(source.arg, data);\n+\tdata->data = NULL;\n+\tdata->len = 0;\n+\tdata->source.ops = NULL;\n+\tdata->source.arg = NULL;\n }\n \n void block_source_close(struct reftable_block_source *source)\n@@ -34,11 +34,11 @@ void block_source_close(struct reftable_block_source *source)\n \tsource->ops = NULL;\n }\n \n-ssize_t block_source_read_block(struct reftable_block_source *source,\n-\t\t\t\tstruct reftable_block *dest, uint64_t off,\n-\t\t\t\tuint32_t size)\n+ssize_t block_source_read_data(struct reftable_block_source *source,\n+\t\t\t       struct reftable_block_data *dest, uint64_t off,\n+\t\t\t       uint32_t size)\n {\n-\tssize_t result = source->ops->read_block(source->arg, dest, off, size);\n+\tssize_t result = source->ops->read_data(source->arg, dest, off, size);\n \tdest->source = *source;\n \treturn result;\n }\n@@ -48,7 +48,7 @@ uint64_t block_source_size(struct reftable_block_source *source)\n \treturn source->ops->size(source->arg);\n }\n \n-static void reftable_buf_return_block(void *b REFTABLE_UNUSED, struct reftable_block *dest)\n+static void reftable_buf_release_data(void *b REFTABLE_UNUSED, struct reftable_block_data *dest)\n {\n \tif (dest->len)\n \t\tmemset(dest->data, 0xff, dest->len);\n@@ -59,8 +59,8 @@ static void reftable_buf_close(void *b REFTABLE_UNUSED)\n {\n }\n \n-static ssize_t reftable_buf_read_block(void *v, struct reftable_block *dest,\n-\t\t\t\t       uint64_t off, uint32_t size)\n+static ssize_t reftable_buf_read_data(void *v, struct reftable_block_data *dest,\n+\t\t\t\t      uint64_t off, uint32_t size)\n {\n \tstruct reftable_buf *b = v;\n \tassert(off + size <= b->len);\n@@ -79,8 +79,8 @@ static uint64_t reftable_buf_size(void *b)\n \n static struct reftable_block_source_vtable reftable_buf_vtable = {\n \t.size = &reftable_buf_size,\n-\t.read_block = &reftable_buf_read_block,\n-\t.return_block = &reftable_buf_return_block,\n+\t.read_data = &reftable_buf_read_data,\n+\t.release_data = &reftable_buf_release_data,\n \t.close = &reftable_buf_close,\n };\n \n@@ -102,7 +102,7 @@ static uint64_t file_size(void *b)\n \treturn ((struct file_block_source *)b)->size;\n }\n \n-static void file_return_block(void *b REFTABLE_UNUSED, struct reftable_block *dest REFTABLE_UNUSED)\n+static void file_release_data(void *b REFTABLE_UNUSED, struct reftable_block_data *dest REFTABLE_UNUSED)\n {\n }\n \n@@ -113,8 +113,8 @@ static void file_close(void *v)\n \treftable_free(b);\n }\n \n-static ssize_t file_read_block(void *v, struct reftable_block *dest, uint64_t off,\n-\t\t\t       uint32_t size)\n+static ssize_t file_read_data(void *v, struct reftable_block_data *dest, uint64_t off,\n+\t\t\t      uint32_t size)\n {\n \tstruct file_block_source *b = v;\n \tassert(off + size <= b->size);\n@@ -125,8 +125,8 @@ static ssize_t file_read_block(void *v, struct reftable_block *dest, uint64_t of\n \n static struct reftable_block_source_vtable file_vtable = {\n \t.size = &file_size,\n-\t.read_block = &file_read_block,\n-\t.return_block = &file_return_block,\n+\t.read_data = &file_read_data,\n+\t.release_data = &file_release_data,\n \t.close = &file_close,\n };\n \ndiff --git a/reftable/blocksource.h b/reftable/blocksource.h\nindex 639b9a1a3c5..a110e059580 100644\n--- a/reftable/blocksource.h\n+++ b/reftable/blocksource.h\n@@ -12,7 +12,7 @@\n #include \"system.h\"\n \n struct reftable_block_source;\n-struct reftable_block;\n+struct reftable_block_data;\n struct reftable_buf;\n \n /*\n@@ -24,9 +24,9 @@ void block_source_close(struct reftable_block_source *source);\n /*\n  * Read a block of length `size` from the source at the given `off`.\n  */\n-ssize_t block_source_read_block(struct reftable_block_source *source,\n-\t\t\t\tstruct reftable_block *dest, uint64_t off,\n-\t\t\t\tuint32_t size);\n+ssize_t block_source_read_data(struct reftable_block_source *source,\n+\t\t\t       struct reftable_block_data *dest, uint64_t off,\n+\t\t\t       uint32_t size);\n \n /*\n  * Return the total length of the underlying resource.\n@@ -37,7 +37,7 @@ uint64_t block_source_size(struct reftable_block_source *source);\n  * Return a block to its original source, releasing any resources associated\n  * with it.\n  */\n-void block_source_return_block(struct reftable_block *block);\n+void block_source_release_data(struct reftable_block_data *data);\n \n /* Create an in-memory block source for reading reftables. */\n void block_source_from_buf(struct reftable_block_source *bs,\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex 6af6eb49396..c00ffc88577 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -114,7 +114,7 @@ static void indexed_table_ref_iter_close(void *p)\n {\n \tstruct indexed_table_ref_iter *it = p;\n \tblock_iter_close(&it->cur);\n-\tblock_source_return_block(&it->block_reader.block);\n+\tblock_source_release_data(&it->block_reader.block_data);\n \treftable_free(it->offsets);\n \treftable_buf_release(&it->oid);\n }\n@@ -128,7 +128,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n \t\treturn 1;\n \t}\n \n-\tblock_source_return_block(&it->block_reader.block);\n+\tblock_source_release_data(&it->block_reader.block_data);\n \n \toff = it->offsets[it->offset_idx++];\n \terr = table_init_block_reader(it->table, &it->block_reader, off,\ndiff --git a/reftable/reftable-blocksource.h b/reftable/reftable-blocksource.h\nindex 96430b629e4..f5ba867bd60 100644\n--- a/reftable/reftable-blocksource.h\n+++ b/reftable/reftable-blocksource.h\n@@ -21,7 +21,7 @@ struct reftable_block_source {\n \n /* a contiguous segment of bytes. It keeps track of its generating block_source\n  * so it can return itself into the pool. */\n-struct reftable_block {\n+struct reftable_block_data {\n \tuint8_t *data;\n \tsize_t len;\n \tstruct reftable_block_source source;\n@@ -29,20 +29,20 @@ struct reftable_block {\n \n /* block_source_vtable are the operations that make up block_source */\n struct reftable_block_source_vtable {\n-\t/* returns the size of a block source */\n+\t/* Returns the size of a block source. */\n \tuint64_t (*size)(void *source);\n \n \t/*\n \t * Reads a segment from the block source. It is an error to read beyond\n \t * the end of the block.\n \t */\n-\tssize_t (*read_block)(void *source, struct reftable_block *dest,\n-\t\t\t     uint64_t off, uint32_t size);\n+\tssize_t (*read_data)(void *source, struct reftable_block_data *dest,\n+\t\t\tuint64_t off, uint32_t size);\n \n-\t/* mark the block as read; may return the data back to malloc */\n-\tvoid (*return_block)(void *source, struct reftable_block *blockp);\n+\t/* Mark the block as read; may release the data. */\n+\tvoid (*release_data)(void *source, struct reftable_block_data *data);\n \n-\t/* release all resources associated with the block source */\n+\t/* Release all resources associated with the block source. */\n \tvoid (*close)(void *source);\n };\n \ndiff --git a/reftable/table.c b/reftable/table.c\nindex 7c0f1c9e6e3..c86b1d4c649 100644\n--- a/reftable/table.c\n+++ b/reftable/table.c\n@@ -320,7 +320,7 @@ static int table_iter_seek_linear(struct table_iter *ti,\n \t\t * as we have more than three blocks we would have an index, so\n \t\t * we would not do a linear search there anymore.\n \t\t */\n-\t\tmemset(&next.br.block, 0, sizeof(next.br.block));\n+\t\tmemset(&next.br.block_data, 0, sizeof(next.br.block_data));\n \t\tnext.br.zstream = NULL;\n \t\tnext.br.uncompressed_data = NULL;\n \t\tnext.br.uncompressed_cap = 0;\n@@ -526,8 +526,8 @@ int reftable_table_init_log_iterator(struct reftable_table *t,\n int reftable_table_new(struct reftable_table **out,\n \t\t       struct reftable_block_source *source, char const *name)\n {\n-\tstruct reftable_block footer = { 0 };\n-\tstruct reftable_block header = { 0 };\n+\tstruct reftable_block_data footer = { 0 };\n+\tstruct reftable_block_data header = { 0 };\n \tstruct reftable_table *t;\n \tuint64_t file_size = block_source_size(source);\n \tuint32_t read_size;\n@@ -550,7 +550,7 @@ int reftable_table_new(struct reftable_table **out,\n \t\tgoto done;\n \t}\n \n-\tbytes_read = block_source_read_block(source, &header, 0, read_size);\n+\tbytes_read = block_source_read_data(source, &header, 0, read_size);\n \tif (bytes_read < 0 || (size_t)bytes_read != read_size) {\n \t\terr = REFTABLE_IO_ERROR;\n \t\tgoto done;\n@@ -576,8 +576,8 @@ int reftable_table_new(struct reftable_table **out,\n \tt->hash_id = 0;\n \tt->refcount = 1;\n \n-\tbytes_read = block_source_read_block(source, &footer, t->size,\n-\t\t\t\t\t     footer_size(t->version));\n+\tbytes_read = block_source_read_data(source, &footer, t->size,\n+\t\t\t\t\t    footer_size(t->version));\n \tif (bytes_read < 0 || (size_t)bytes_read != footer_size(t->version)) {\n \t\terr = REFTABLE_IO_ERROR;\n \t\tgoto done;\n@@ -590,8 +590,8 @@ int reftable_table_new(struct reftable_table **out,\n \t*out = t;\n \n done:\n-\tblock_source_return_block(&footer);\n-\tblock_source_return_block(&header);\n+\tblock_source_release_data(&footer);\n+\tblock_source_release_data(&header);\n \tif (err) {\n \t\tif (t)\n \t\t\treftable_free(t->name);\ndiff --git a/t/unit-tests/t-reftable-readwrite.c b/t/unit-tests/t-reftable-readwrite.c\nindex 3fba888cdaa..4c49129439e 100644\n--- a/t/unit-tests/t-reftable-readwrite.c\n+++ b/t/unit-tests/t-reftable-readwrite.c\n@@ -23,22 +23,22 @@ static void t_buffer(void)\n {\n \tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n \tstruct reftable_block_source source = { 0 };\n-\tstruct reftable_block out = { 0 };\n+\tstruct reftable_block_data out = { 0 };\n \tint n;\n \tuint8_t in[] = \"hello\";\n \tcheck(!reftable_buf_add(&buf, in, sizeof(in)));\n \tblock_source_from_buf(&source, &buf);\n \tcheck_int(block_source_size(&source), ==, 6);\n-\tn = block_source_read_block(&source, &out, 0, sizeof(in));\n+\tn = block_source_read_data(&source, &out, 0, sizeof(in));\n \tcheck_int(n, ==, sizeof(in));\n \tcheck(!memcmp(in, out.data, n));\n-\tblock_source_return_block(&out);\n+\tblock_source_release_data(&out);\n \n-\tn = block_source_read_block(&source, &out, 1, 2);\n+\tn = block_source_read_data(&source, &out, 1, 2);\n \tcheck_int(n, ==, 2);\n \tcheck(!memcmp(out.data, \"el\", 2));\n \n-\tblock_source_return_block(&out);\n+\tblock_source_release_data(&out);\n \tblock_source_close(&source);\n \treftable_buf_release(&buf);\n }\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515768","messageId":"20250407-pks-reftable-polishing-v2-5-316c4ff10571@pks.im","threadId":"63222","inReplyTo":"20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im","subject":"[PATCH v2 05/16] reftable/table: move reading block into block reader","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T13:16:17Z","receivedAt":"2025-04-07T13:16:28Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The logic to read blocks from a reftable is scattered across both the\ntable and the block subsystems. Besides causing somewhat fuzzy\nresponsibilities, it also means that we have to awkwardly pass around\nthe ownership of blocks between the subsystems.\n\nRefactor the code so that we stop passing the block when initializing a\nreader, but instead by passing in the block source plus the offset at\nwhich we're supposed to read a block. Like this, the ownership of the\nblock itself doesn't need to get handed over as the block reader is the\none owning the block right from the start.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/block.c                | 87 ++++++++++++++++++++++++++---------------\n reftable/block.h                |  8 ++--\n reftable/table.c                | 65 +++---------------------------\n t/unit-tests/t-reftable-block.c | 76 ++++++++++++++++++-----------------\n 4 files changed, 107 insertions(+), 129 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex f2567a8f0fd..2517108b8ef 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -209,31 +209,57 @@ int block_writer_finish(struct block_writer *w)\n \treturn w->next;\n }\n \n-int block_reader_init(struct block_reader *br, struct reftable_block *block,\n-\t\t      uint32_t header_off, uint32_t table_block_size,\n-\t\t      uint32_t hash_size)\n+static int read_block(struct reftable_block_source *source,\n+\t\t      struct reftable_block *dest, uint64_t off,\n+\t\t      uint32_t sz)\n {\n+\tsize_t size = block_source_size(source);\n+\tblock_source_return_block(dest);\n+\tif (off >= size)\n+\t\treturn 0;\n+\tif (off + sz > size)\n+\t\tsz = size - off;\n+\treturn block_source_read_block(source, dest, off, sz);\n+}\n+\n+int block_reader_init(struct block_reader *br,\n+\t\t      struct reftable_block_source *source,\n+\t\t      uint32_t offset, uint32_t header_size,\n+\t\t      uint32_t table_block_size, uint32_t hash_size)\n+{\n+\tuint32_t guess_block_size = table_block_size ?\n+\t\ttable_block_size : DEFAULT_BLOCK_SIZE;\n \tuint32_t full_block_size = table_block_size;\n-\tuint8_t typ = block->data[header_off];\n-\tuint32_t sz = reftable_get_be24(block->data + header_off + 1);\n \tuint16_t restart_count;\n \tuint32_t restart_off;\n+\tuint32_t block_size;\n+\tuint8_t block_type;\n \tint err;\n \n-\tblock_source_return_block(&br->block);\n+\terr = read_block(source, &br->block, offset, guess_block_size);\n+\tif (err < 0)\n+\t\tgoto done;\n \n-\tif (!reftable_is_block_type(typ)) {\n-\t\terr =  REFTABLE_FORMAT_ERROR;\n+\tblock_type = br->block.data[header_size];\n+\tif (!reftable_is_block_type(block_type)) {\n+\t\terr = REFTABLE_FORMAT_ERROR;\n \t\tgoto done;\n \t}\n \n-\tif (typ == BLOCK_TYPE_LOG) {\n-\t\tuint32_t block_header_skip = 4 + header_off;\n-\t\tuLong dst_len = sz - block_header_skip;\n-\t\tuLong src_len = block->len - block_header_skip;\n+\tblock_size = reftable_get_be24(br->block.data + header_size + 1);\n+\tif (block_size > guess_block_size) {\n+\t\terr = read_block(source, &br->block, offset, block_size);\n+\t\tif (err < 0)\n+\t\t\tgoto done;\n+\t}\n+\n+\tif (block_type == BLOCK_TYPE_LOG) {\n+\t\tuint32_t block_header_skip = 4 + header_size;\n+\t\tuLong dst_len = block_size - block_header_skip;\n+\t\tuLong src_len = br->block.len - block_header_skip;\n \n \t\t/* Log blocks specify the *uncompressed* size in their header. */\n-\t\tREFTABLE_ALLOC_GROW_OR_NULL(br->uncompressed_data, sz,\n+\t\tREFTABLE_ALLOC_GROW_OR_NULL(br->uncompressed_data, block_size,\n \t\t\t\t\t    br->uncompressed_cap);\n \t\tif (!br->uncompressed_data) {\n \t\t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n@@ -241,7 +267,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\t}\n \n \t\t/* Copy over the block header verbatim. It's not compressed. */\n-\t\tmemcpy(br->uncompressed_data, block->data, block_header_skip);\n+\t\tmemcpy(br->uncompressed_data, br->block.data, block_header_skip);\n \n \t\tif (!br->zstream) {\n \t\t\tREFTABLE_CALLOC_ARRAY(br->zstream, 1);\n@@ -259,7 +285,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\t\tgoto done;\n \t\t}\n \n-\t\tbr->zstream->next_in = block->data + block_header_skip;\n+\t\tbr->zstream->next_in = br->block.data + block_header_skip;\n \t\tbr->zstream->avail_in = src_len;\n \t\tbr->zstream->next_out = br->uncompressed_data + block_header_skip;\n \t\tbr->zstream->avail_out = dst_len;\n@@ -278,43 +304,41 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\t}\n \t\terr = 0;\n \n-\t\tif (br->zstream->total_out + block_header_skip != sz) {\n+\t\tif (br->zstream->total_out + block_header_skip != block_size) {\n \t\t\terr = REFTABLE_FORMAT_ERROR;\n \t\t\tgoto done;\n \t\t}\n \n \t\t/* We're done with the input data. */\n-\t\tblock_source_return_block(block);\n-\t\tblock->data = br->uncompressed_data;\n-\t\tblock->len = sz;\n+\t\tblock_source_return_block(&br->block);\n+\t\tbr->block.data = br->uncompressed_data;\n+\t\tbr->block.len = block_size;\n \t\tfull_block_size = src_len + block_header_skip - br->zstream->avail_in;\n \t} else if (full_block_size == 0) {\n-\t\tfull_block_size = sz;\n-\t} else if (sz < full_block_size && sz < block->len &&\n-\t\t   block->data[sz] != 0) {\n+\t\tfull_block_size = block_size;\n+\t} else if (block_size < full_block_size && block_size < br->block.len &&\n+\t\t   br->block.data[block_size] != 0) {\n \t\t/* If the block is smaller than the full block size, it is\n \t\t   padded (data followed by '\\0') or the next block is\n \t\t   unaligned. */\n-\t\tfull_block_size = sz;\n+\t\tfull_block_size = block_size;\n \t}\n \n-\trestart_count = reftable_get_be16(block->data + sz - 2);\n-\trestart_off = sz - 2 - 3 * restart_count;\n-\n-\t/* transfer ownership. */\n-\tbr->block = *block;\n-\tblock->data = NULL;\n-\tblock->len = 0;\n+\trestart_count = reftable_get_be16(br->block.data + block_size - 2);\n+\trestart_off = block_size - 2 - 3 * restart_count;\n \n+\tbr->block_type = block_type;\n \tbr->hash_size = hash_size;\n \tbr->restart_off = restart_off;\n \tbr->full_block_size = full_block_size;\n-\tbr->header_off = header_off;\n+\tbr->header_off = header_size;\n \tbr->restart_count = restart_count;\n \n \terr = 0;\n \n done:\n+\tif (err < 0)\n+\t\tblock_reader_release(br);\n \treturn err;\n }\n \n@@ -324,6 +348,7 @@ void block_reader_release(struct block_reader *br)\n \treftable_free(br->zstream);\n \treftable_free(br->uncompressed_data);\n \tblock_source_return_block(&br->block);\n+\tmemset(br, 0, sizeof(*br));\n }\n \n uint8_t block_reader_type(const struct block_reader *r)\ndiff --git a/reftable/block.h b/reftable/block.h\nindex b78f322e646..6afb1b2952a 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -89,12 +89,14 @@ struct block_reader {\n \t/* size of the data in the file. For log blocks, this is the compressed\n \t * size. */\n \tuint32_t full_block_size;\n+\tuint8_t block_type;\n };\n \n /* initializes a block reader. */\n-int block_reader_init(struct block_reader *br, struct reftable_block *bl,\n-\t\t      uint32_t header_off, uint32_t table_block_size,\n-\t\t      uint32_t hash_size);\n+int block_reader_init(struct block_reader *br,\n+\t\t      struct reftable_block_source *source,\n+\t\t      uint32_t offset, uint32_t header_size,\n+\t\t      uint32_t table_block_size, uint32_t hash_size);\n \n void block_reader_release(struct block_reader *br);\n \ndiff --git a/reftable/table.c b/reftable/table.c\nindex ec84545707c..7c0f1c9e6e3 100644\n--- a/reftable/table.c\n+++ b/reftable/table.c\n@@ -30,23 +30,6 @@ table_offsets_for(struct reftable_table *t, uint8_t typ)\n \tabort();\n }\n \n-static int table_get_block(struct reftable_table *t,\n-\t\t\t   struct reftable_block *dest, uint64_t off,\n-\t\t\t   uint32_t sz)\n-{\n-\tssize_t bytes_read;\n-\tif (off >= t->size)\n-\t\treturn 0;\n-\tif (off + sz > t->size)\n-\t\tsz = t->size - off;\n-\n-\tbytes_read = block_source_read_block(&t->source, dest, off, sz);\n-\tif (bytes_read < 0)\n-\t\treturn (int)bytes_read;\n-\n-\treturn 0;\n-}\n-\n enum reftable_hash reftable_table_hash_id(struct reftable_table *t)\n {\n \treturn t->hash_id;\n@@ -180,64 +163,28 @@ static void table_iter_block_done(struct table_iter *ti)\n \tblock_iter_reset(&ti->bi);\n }\n \n-static int32_t extract_block_size(uint8_t *data, uint8_t *typ, uint64_t off,\n-\t\t\t\t  int version)\n-{\n-\tint32_t result = 0;\n-\n-\tif (off == 0) {\n-\t\tdata += header_size(version);\n-\t}\n-\n-\t*typ = data[0];\n-\tif (reftable_is_block_type(*typ)) {\n-\t\tresult = reftable_get_be24(data + 1);\n-\t}\n-\treturn result;\n-}\n-\n int table_init_block_reader(struct reftable_table *t, struct block_reader *br,\n \t\t\t    uint64_t next_off, uint8_t want_typ)\n {\n-\tint32_t guess_block_size = t->block_size ? t->block_size :\n-\t\t\t\t\t\t\t DEFAULT_BLOCK_SIZE;\n-\tstruct reftable_block block = { NULL };\n-\tuint8_t block_typ = 0;\n-\tint err = 0;\n \tuint32_t header_off = next_off ? 0 : header_size(t->version);\n-\tint32_t block_size = 0;\n+\tint err;\n \n \tif (next_off >= t->size)\n \t\treturn 1;\n \n-\terr = table_get_block(t, &block, next_off, guess_block_size);\n+\terr = block_reader_init(br, &t->source, next_off, header_off,\n+\t\t\t\tt->block_size, hash_size(t->hash_id));\n \tif (err < 0)\n \t\tgoto done;\n \n-\tblock_size = extract_block_size(block.data, &block_typ, next_off,\n-\t\t\t\t\tt->version);\n-\tif (block_size < 0) {\n-\t\terr = block_size;\n-\t\tgoto done;\n-\t}\n-\tif (want_typ != BLOCK_TYPE_ANY && block_typ != want_typ) {\n+\tif (want_typ != BLOCK_TYPE_ANY && br->block_type != want_typ) {\n \t\terr = 1;\n \t\tgoto done;\n \t}\n \n-\tif (block_size > guess_block_size) {\n-\t\tblock_source_return_block(&block);\n-\t\terr = table_get_block(t, &block, next_off, block_size);\n-\t\tif (err < 0) {\n-\t\t\tgoto done;\n-\t\t}\n-\t}\n-\n-\terr = block_reader_init(br, &block, header_off, t->block_size,\n-\t\t\t\thash_size(t->hash_id));\n done:\n-\tblock_source_return_block(&block);\n-\n+\tif (err)\n+\t\tblock_reader_release(br);\n \treturn err;\n }\n \ndiff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c\nindex 8bb40482347..dbec6a5d0d0 100644\n--- a/t/unit-tests/t-reftable-block.c\n+++ b/t/unit-tests/t-reftable-block.c\n@@ -19,7 +19,7 @@ static void t_ref_block_read_write(void)\n \tstruct reftable_record recs[30];\n \tconst size_t N = ARRAY_SIZE(recs);\n \tconst size_t block_size = 1024;\n-\tstruct reftable_block block = { 0 };\n+\tstruct reftable_block_source source = { 0 };\n \tstruct block_writer bw = {\n \t\t.last_key = REFTABLE_BUF_INIT,\n \t};\n@@ -30,13 +30,14 @@ static void t_ref_block_read_write(void)\n \tint ret;\n \tstruct block_reader br = { 0 };\n \tstruct block_iter it = BLOCK_ITER_INIT;\n-\tstruct reftable_buf want = REFTABLE_BUF_INIT, buf = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf want = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf block = REFTABLE_BUF_INIT;\n \n-\tREFTABLE_CALLOC_ARRAY(block.data, block_size);\n-\tcheck(block.data != NULL);\n+\tREFTABLE_CALLOC_ARRAY(block.buf, block_size);\n+\tcheck(block.buf != NULL);\n \tblock.len = block_size;\n-\tblock_source_from_buf(&block.source ,&buf);\n-\tret = block_writer_init(&bw, BLOCK_TYPE_REF, block.data, block_size,\n+\n+\tret = block_writer_init(&bw, BLOCK_TYPE_REF, (uint8_t *) block.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -62,7 +63,8 @@ static void t_ref_block_read_write(void)\n \n \tblock_writer_release(&bw);\n \n-\tblock_reader_init(&br, &block, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n+\tblock_source_from_buf(&source ,&block);\n+\tblock_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n \tblock_iter_seek_start(&it, &br);\n \n@@ -100,9 +102,8 @@ static void t_ref_block_read_write(void)\n \tblock_reader_release(&br);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n-\tblock_source_return_block(&br.block);\n \treftable_buf_release(&want);\n-\treftable_buf_release(&buf);\n+\treftable_buf_release(&block);\n \tfor (i = 0; i < N; i++)\n \t\treftable_record_release(&recs[i]);\n }\n@@ -113,7 +114,7 @@ static void t_log_block_read_write(void)\n \tstruct reftable_record recs[30];\n \tconst size_t N = ARRAY_SIZE(recs);\n \tconst size_t block_size = 2048;\n-\tstruct reftable_block block = { 0 };\n+\tstruct reftable_block_source source = { 0 };\n \tstruct block_writer bw = {\n \t\t.last_key = REFTABLE_BUF_INIT,\n \t};\n@@ -124,13 +125,14 @@ static void t_log_block_read_write(void)\n \tint ret;\n \tstruct block_reader br = { 0 };\n \tstruct block_iter it = BLOCK_ITER_INIT;\n-\tstruct reftable_buf want = REFTABLE_BUF_INIT, buf = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf want = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf block = REFTABLE_BUF_INIT;\n \n-\tREFTABLE_CALLOC_ARRAY(block.data, block_size);\n-\tcheck(block.data != NULL);\n+\tREFTABLE_CALLOC_ARRAY(block.buf, block_size);\n+\tcheck(block.buf != NULL);\n \tblock.len = block_size;\n-\tblock_source_from_buf(&block.source ,&buf);\n-\tret = block_writer_init(&bw, BLOCK_TYPE_LOG, block.data, block_size,\n+\n+\tret = block_writer_init(&bw, BLOCK_TYPE_LOG, (uint8_t *) block.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -151,7 +153,8 @@ static void t_log_block_read_write(void)\n \n \tblock_writer_release(&bw);\n \n-\tblock_reader_init(&br, &block, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n+\tblock_source_from_buf(&source, &block);\n+\tblock_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n \tblock_iter_seek_start(&it, &br);\n \n@@ -190,9 +193,8 @@ static void t_log_block_read_write(void)\n \tblock_reader_release(&br);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n-\tblock_source_return_block(&br.block);\n \treftable_buf_release(&want);\n-\treftable_buf_release(&buf);\n+\treftable_buf_release(&block);\n \tfor (i = 0; i < N; i++)\n \t\treftable_record_release(&recs[i]);\n }\n@@ -203,7 +205,7 @@ static void t_obj_block_read_write(void)\n \tstruct reftable_record recs[30];\n \tconst size_t N = ARRAY_SIZE(recs);\n \tconst size_t block_size = 1024;\n-\tstruct reftable_block block = { 0 };\n+\tstruct reftable_block_source source = { 0 };\n \tstruct block_writer bw = {\n \t\t.last_key = REFTABLE_BUF_INIT,\n \t};\n@@ -214,13 +216,14 @@ static void t_obj_block_read_write(void)\n \tint ret;\n \tstruct block_reader br = { 0 };\n \tstruct block_iter it = BLOCK_ITER_INIT;\n-\tstruct reftable_buf want = REFTABLE_BUF_INIT, buf = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf want = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf block = REFTABLE_BUF_INIT;\n \n-\tREFTABLE_CALLOC_ARRAY(block.data, block_size);\n-\tcheck(block.data != NULL);\n+\tREFTABLE_CALLOC_ARRAY(block.buf, block_size);\n+\tcheck(block.buf != NULL);\n \tblock.len = block_size;\n-\tblock_source_from_buf(&block.source, &buf);\n-\tret = block_writer_init(&bw, BLOCK_TYPE_OBJ, block.data, block_size,\n+\n+\tret = block_writer_init(&bw, BLOCK_TYPE_OBJ, (uint8_t *) block.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -243,7 +246,8 @@ static void t_obj_block_read_write(void)\n \n \tblock_writer_release(&bw);\n \n-\tblock_reader_init(&br, &block, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n+\tblock_source_from_buf(&source, &block);\n+\tblock_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n \tblock_iter_seek_start(&it, &br);\n \n@@ -273,9 +277,8 @@ static void t_obj_block_read_write(void)\n \tblock_reader_release(&br);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n-\tblock_source_return_block(&br.block);\n \treftable_buf_release(&want);\n-\treftable_buf_release(&buf);\n+\treftable_buf_release(&block);\n \tfor (i = 0; i < N; i++)\n \t\treftable_record_release(&recs[i]);\n }\n@@ -286,7 +289,7 @@ static void t_index_block_read_write(void)\n \tstruct reftable_record recs[30];\n \tconst size_t N = ARRAY_SIZE(recs);\n \tconst size_t block_size = 1024;\n-\tstruct reftable_block block = { 0 };\n+\tstruct reftable_block_source source = { 0 };\n \tstruct block_writer bw = {\n \t\t.last_key = REFTABLE_BUF_INIT,\n \t};\n@@ -298,13 +301,14 @@ static void t_index_block_read_write(void)\n \tint ret;\n \tstruct block_reader br = { 0 };\n \tstruct block_iter it = BLOCK_ITER_INIT;\n-\tstruct reftable_buf want = REFTABLE_BUF_INIT, buf = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf want = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf block = REFTABLE_BUF_INIT;\n \n-\tREFTABLE_CALLOC_ARRAY(block.data, block_size);\n-\tcheck(block.data != NULL);\n+\tREFTABLE_CALLOC_ARRAY(block.buf, block_size);\n+\tcheck(block.buf != NULL);\n \tblock.len = block_size;\n-\tblock_source_from_buf(&block.source, &buf);\n-\tret = block_writer_init(&bw, BLOCK_TYPE_INDEX, block.data, block_size,\n+\n+\tret = block_writer_init(&bw, BLOCK_TYPE_INDEX, (uint8_t *) block.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -327,7 +331,8 @@ static void t_index_block_read_write(void)\n \n \tblock_writer_release(&bw);\n \n-\tblock_reader_init(&br, &block, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n+\tblock_source_from_buf(&source, &block);\n+\tblock_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n \tblock_iter_seek_start(&it, &br);\n \n@@ -365,9 +370,8 @@ static void t_index_block_read_write(void)\n \tblock_reader_release(&br);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n-\tblock_source_return_block(&br.block);\n \treftable_buf_release(&want);\n-\treftable_buf_release(&buf);\n+\treftable_buf_release(&block);\n \tfor (i = 0; i < N; i++)\n \t\treftable_record_release(&recs[i]);\n }\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515770","messageId":"20250407-pks-reftable-polishing-v2-7-316c4ff10571@pks.im","threadId":"63222","inReplyTo":"20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im","subject":"[PATCH v2 07/16] reftable/block: rename `block_reader` to `reftable_block`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T13:16:19Z","receivedAt":"2025-04-07T13:16:29Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The `block_reader` structure is used to access parsed data of a reftable\nblock. The structure is currently treated as an internal implementation\ndetail and not exposed via our public interfaces. The functionality\nprovided by the structure is useful to external users of the reftable\nlibrary though, for example when implementing consistency checks that\nneed to scan through the blocks manually.\n\nRename the structure to `reftable_block` now that the name has been made\navailable in the preceding commit. This name is in line with the naming\nschema used for other data structures like `reftable_table` in that it\ndescribes the underlying entity that it provides access to.\n\nThe new data structure isn't yet exposed via the public interface, which\nis left for a subsequent commit.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/block.c                | 142 ++++++++++++++++++++--------------------\n reftable/block.h                |  29 ++++----\n reftable/iter.c                 |   9 ++-\n reftable/iter.h                 |   2 +-\n reftable/table.c                |  46 ++++++-------\n reftable/table.h                |   8 ++-\n t/unit-tests/t-reftable-block.c | 102 ++++++++++++++---------------\n 7 files changed, 172 insertions(+), 166 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex dca30b8527f..844da397833 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -222,10 +222,10 @@ static int read_block(struct reftable_block_source *source,\n \treturn block_source_read_data(source, dest, off, sz);\n }\n \n-int block_reader_init(struct block_reader *br,\n-\t\t      struct reftable_block_source *source,\n-\t\t      uint32_t offset, uint32_t header_size,\n-\t\t      uint32_t table_block_size, uint32_t hash_size)\n+int reftable_block_init(struct reftable_block *block,\n+\t\t\tstruct reftable_block_source *source,\n+\t\t\tuint32_t offset, uint32_t header_size,\n+\t\t\tuint32_t table_block_size, uint32_t hash_size)\n {\n \tuint32_t guess_block_size = table_block_size ?\n \t\ttable_block_size : DEFAULT_BLOCK_SIZE;\n@@ -236,19 +236,19 @@ int block_reader_init(struct block_reader *br,\n \tuint8_t block_type;\n \tint err;\n \n-\terr = read_block(source, &br->block_data, offset, guess_block_size);\n+\terr = read_block(source, &block->block_data, offset, guess_block_size);\n \tif (err < 0)\n \t\tgoto done;\n \n-\tblock_type = br->block_data.data[header_size];\n+\tblock_type = block->block_data.data[header_size];\n \tif (!reftable_is_block_type(block_type)) {\n \t\terr = REFTABLE_FORMAT_ERROR;\n \t\tgoto done;\n \t}\n \n-\tblock_size = reftable_get_be24(br->block_data.data + header_size + 1);\n+\tblock_size = reftable_get_be24(block->block_data.data + header_size + 1);\n \tif (block_size > guess_block_size) {\n-\t\terr = read_block(source, &br->block_data, offset, block_size);\n+\t\terr = read_block(source, &block->block_data, offset, block_size);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \t}\n@@ -256,39 +256,39 @@ int block_reader_init(struct block_reader *br,\n \tif (block_type == BLOCK_TYPE_LOG) {\n \t\tuint32_t block_header_skip = 4 + header_size;\n \t\tuLong dst_len = block_size - block_header_skip;\n-\t\tuLong src_len = br->block_data.len - block_header_skip;\n+\t\tuLong src_len = block->block_data.len - block_header_skip;\n \n \t\t/* Log blocks specify the *uncompressed* size in their header. */\n-\t\tREFTABLE_ALLOC_GROW_OR_NULL(br->uncompressed_data, block_size,\n-\t\t\t\t\t    br->uncompressed_cap);\n-\t\tif (!br->uncompressed_data) {\n+\t\tREFTABLE_ALLOC_GROW_OR_NULL(block->uncompressed_data, block_size,\n+\t\t\t\t\t    block->uncompressed_cap);\n+\t\tif (!block->uncompressed_data) {\n \t\t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\t\tgoto done;\n \t\t}\n \n \t\t/* Copy over the block header verbatim. It's not compressed. */\n-\t\tmemcpy(br->uncompressed_data, br->block_data.data, block_header_skip);\n+\t\tmemcpy(block->uncompressed_data, block->block_data.data, block_header_skip);\n \n-\t\tif (!br->zstream) {\n-\t\t\tREFTABLE_CALLOC_ARRAY(br->zstream, 1);\n-\t\t\tif (!br->zstream) {\n+\t\tif (!block->zstream) {\n+\t\t\tREFTABLE_CALLOC_ARRAY(block->zstream, 1);\n+\t\t\tif (!block->zstream) {\n \t\t\t\terr = REFTABLE_OUT_OF_MEMORY_ERROR;\n \t\t\t\tgoto done;\n \t\t\t}\n \n-\t\t\terr = inflateInit(br->zstream);\n+\t\t\terr = inflateInit(block->zstream);\n \t\t} else {\n-\t\t\terr = inflateReset(br->zstream);\n+\t\t\terr = inflateReset(block->zstream);\n \t\t}\n \t\tif (err != Z_OK) {\n \t\t\terr = REFTABLE_ZLIB_ERROR;\n \t\t\tgoto done;\n \t\t}\n \n-\t\tbr->zstream->next_in = br->block_data.data + block_header_skip;\n-\t\tbr->zstream->avail_in = src_len;\n-\t\tbr->zstream->next_out = br->uncompressed_data + block_header_skip;\n-\t\tbr->zstream->avail_out = dst_len;\n+\t\tblock->zstream->next_in = block->block_data.data + block_header_skip;\n+\t\tblock->zstream->avail_in = src_len;\n+\t\tblock->zstream->next_out = block->uncompressed_data + block_header_skip;\n+\t\tblock->zstream->avail_out = dst_len;\n \n \t\t/*\n \t\t * We know both input as well as output size, and we know that\n@@ -297,71 +297,71 @@ int block_reader_init(struct block_reader *br,\n \t\t * here to instruct zlib to inflate the data in one go, which\n \t\t * is more efficient than using `Z_NO_FLUSH`.\n \t\t */\n-\t\terr = inflate(br->zstream, Z_FINISH);\n+\t\terr = inflate(block->zstream, Z_FINISH);\n \t\tif (err != Z_STREAM_END) {\n \t\t\terr = REFTABLE_ZLIB_ERROR;\n \t\t\tgoto done;\n \t\t}\n \t\terr = 0;\n \n-\t\tif (br->zstream->total_out + block_header_skip != block_size) {\n+\t\tif (block->zstream->total_out + block_header_skip != block_size) {\n \t\t\terr = REFTABLE_FORMAT_ERROR;\n \t\t\tgoto done;\n \t\t}\n \n \t\t/* We're done with the input data. */\n-\t\tblock_source_release_data(&br->block_data);\n-\t\tbr->block_data.data = br->uncompressed_data;\n-\t\tbr->block_data.len = block_size;\n-\t\tfull_block_size = src_len + block_header_skip - br->zstream->avail_in;\n+\t\tblock_source_release_data(&block->block_data);\n+\t\tblock->block_data.data = block->uncompressed_data;\n+\t\tblock->block_data.len = block_size;\n+\t\tfull_block_size = src_len + block_header_skip - block->zstream->avail_in;\n \t} else if (full_block_size == 0) {\n \t\tfull_block_size = block_size;\n-\t} else if (block_size < full_block_size && block_size < br->block_data.len &&\n-\t\t   br->block_data.data[block_size] != 0) {\n+\t} else if (block_size < full_block_size && block_size < block->block_data.len &&\n+\t\t   block->block_data.data[block_size] != 0) {\n \t\t/* If the block is smaller than the full block size, it is\n \t\t   padded (data followed by '\\0') or the next block is\n \t\t   unaligned. */\n \t\tfull_block_size = block_size;\n \t}\n \n-\trestart_count = reftable_get_be16(br->block_data.data + block_size - 2);\n+\trestart_count = reftable_get_be16(block->block_data.data + block_size - 2);\n \trestart_off = block_size - 2 - 3 * restart_count;\n \n-\tbr->block_type = block_type;\n-\tbr->hash_size = hash_size;\n-\tbr->restart_off = restart_off;\n-\tbr->full_block_size = full_block_size;\n-\tbr->header_off = header_size;\n-\tbr->restart_count = restart_count;\n+\tblock->block_type = block_type;\n+\tblock->hash_size = hash_size;\n+\tblock->restart_off = restart_off;\n+\tblock->full_block_size = full_block_size;\n+\tblock->header_off = header_size;\n+\tblock->restart_count = restart_count;\n \n \terr = 0;\n \n done:\n \tif (err < 0)\n-\t\tblock_reader_release(br);\n+\t\treftable_block_release(block);\n \treturn err;\n }\n \n-void block_reader_release(struct block_reader *br)\n+void reftable_block_release(struct reftable_block *block)\n {\n-\tinflateEnd(br->zstream);\n-\treftable_free(br->zstream);\n-\treftable_free(br->uncompressed_data);\n-\tblock_source_release_data(&br->block_data);\n-\tmemset(br, 0, sizeof(*br));\n+\tinflateEnd(block->zstream);\n+\treftable_free(block->zstream);\n+\treftable_free(block->uncompressed_data);\n+\tblock_source_release_data(&block->block_data);\n+\tmemset(block, 0, sizeof(*block));\n }\n \n-uint8_t block_reader_type(const struct block_reader *r)\n+uint8_t reftable_block_type(const struct reftable_block *b)\n {\n-\treturn r->block_data.data[r->header_off];\n+\treturn b->block_data.data[b->header_off];\n }\n \n-int block_reader_first_key(const struct block_reader *br, struct reftable_buf *key)\n+int reftable_block_first_key(const struct reftable_block *block, struct reftable_buf *key)\n {\n-\tint off = br->header_off + 4, n;\n+\tint off = block->header_off + 4, n;\n \tstruct string_view in = {\n-\t\t.buf = br->block_data.data + off,\n-\t\t.len = br->restart_off - off,\n+\t\t.buf = block->block_data.data + off,\n+\t\t.len = block->restart_off - off,\n \t};\n \tuint8_t extra = 0;\n \n@@ -376,33 +376,33 @@ int block_reader_first_key(const struct block_reader *br, struct reftable_buf *k\n \treturn 0;\n }\n \n-static uint32_t block_reader_restart_offset(const struct block_reader *br, size_t idx)\n+static uint32_t block_restart_offset(const struct reftable_block *b, size_t idx)\n {\n-\treturn reftable_get_be24(br->block_data.data + br->restart_off + 3 * idx);\n+\treturn reftable_get_be24(b->block_data.data + b->restart_off + 3 * idx);\n }\n \n-void block_iter_seek_start(struct block_iter *it, const struct block_reader *br)\n+void block_iter_seek_start(struct block_iter *it, const struct reftable_block *b)\n {\n-\tit->block = br->block_data.data;\n-\tit->block_len = br->restart_off;\n-\tit->hash_size = br->hash_size;\n+\tit->block = b->block_data.data;\n+\tit->block_len = b->restart_off;\n+\tit->hash_size = b->hash_size;\n \treftable_buf_reset(&it->last_key);\n-\tit->next_off = br->header_off + 4;\n+\tit->next_off = b->header_off + 4;\n }\n \n struct restart_needle_less_args {\n \tint error;\n \tstruct reftable_buf needle;\n-\tconst struct block_reader *reader;\n+\tconst struct reftable_block *block;\n };\n \n static int restart_needle_less(size_t idx, void *_args)\n {\n \tstruct restart_needle_less_args *args = _args;\n-\tuint32_t off = block_reader_restart_offset(args->reader, idx);\n+\tuint32_t off = block_restart_offset(args->block, idx);\n \tstruct string_view in = {\n-\t\t.buf = args->reader->block_data.data + off,\n-\t\t.len = args->reader->restart_off - off,\n+\t\t.buf = args->block->block_data.data + off,\n+\t\t.len = args->block->restart_off - off,\n \t};\n \tuint64_t prefix_len, suffix_len;\n \tuint8_t extra;\n@@ -477,12 +477,12 @@ void block_iter_close(struct block_iter *it)\n \treftable_buf_release(&it->scratch);\n }\n \n-int block_iter_seek_key(struct block_iter *it, const struct block_reader *br,\n+int block_iter_seek_key(struct block_iter *it, const struct reftable_block *block,\n \t\t\tstruct reftable_buf *want)\n {\n \tstruct restart_needle_less_args args = {\n \t\t.needle = *want,\n-\t\t.reader = br,\n+\t\t.block = block,\n \t};\n \tstruct reftable_record rec;\n \tint err = 0;\n@@ -500,7 +500,7 @@ int block_iter_seek_key(struct block_iter *it, const struct block_reader *br,\n \t * restart point. While that works alright, we would end up scanning\n \t * too many record.\n \t */\n-\ti = binsearch(br->restart_count, &restart_needle_less, &args);\n+\ti = binsearch(block->restart_count, &restart_needle_less, &args);\n \tif (args.error) {\n \t\terr = REFTABLE_FORMAT_ERROR;\n \t\tgoto done;\n@@ -525,21 +525,21 @@ int block_iter_seek_key(struct block_iter *it, const struct block_reader *br,\n \t *     starting from the preceding restart point.\n \t */\n \tif (i > 0)\n-\t\tit->next_off = block_reader_restart_offset(br, i - 1);\n+\t\tit->next_off = block_restart_offset(block, i - 1);\n \telse\n-\t\tit->next_off = br->header_off + 4;\n-\tit->block = br->block_data.data;\n-\tit->block_len = br->restart_off;\n-\tit->hash_size = br->hash_size;\n+\t\tit->next_off = block->header_off + 4;\n+\tit->block = block->block_data.data;\n+\tit->block_len = block->restart_off;\n+\tit->hash_size = block->hash_size;\n \n-\terr = reftable_record_init(&rec, block_reader_type(br));\n+\terr = reftable_record_init(&rec, reftable_block_type(block));\n \tif (err < 0)\n \t\tgoto done;\n \n \t/*\n \t * We're looking for the last entry less than the wanted key so that\n \t * the next call to `block_reader_next()` would yield the wanted\n-\t * record. We thus don't want to position our reader at the sought\n+\t * record. We thus don't want to position our iterator at the sought\n \t * after record, but one before. To do so, we have to go one entry too\n \t * far and then back up.\n \t */\ndiff --git a/reftable/block.h b/reftable/block.h\nindex 948b79e4614..3957aee4293 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -64,8 +64,11 @@ void block_writer_release(struct block_writer *bw);\n \n struct z_stream;\n \n-/* Read a block. */\n-struct block_reader {\n+/*\n+ * A block part of a reftable. Contains records as well as some metadata\n+ * describing them.\n+ */\n+struct reftable_block {\n \t/* offset of the block header; nonzero for the first block in a\n \t * reftable. */\n \tuint32_t header_off;\n@@ -92,19 +95,21 @@ struct block_reader {\n \tuint8_t block_type;\n };\n \n-/* initializes a block reader. */\n-int block_reader_init(struct block_reader *br,\n-\t\t      struct reftable_block_source *source,\n-\t\t      uint32_t offset, uint32_t header_size,\n-\t\t      uint32_t table_block_size, uint32_t hash_size);\n+/*\n+ * Initialize a reftable block from the given block source.\n+ */\n+int reftable_block_init(struct reftable_block *b,\n+\t\t\tstruct reftable_block_source *source,\n+\t\t\tuint32_t offset, uint32_t header_size,\n+\t\t\tuint32_t table_block_size, uint32_t hash_size);\n \n-void block_reader_release(struct block_reader *br);\n+void reftable_block_release(struct reftable_block *b);\n \n /* Returns the block type (eg. 'r' for refs) */\n-uint8_t block_reader_type(const struct block_reader *r);\n+uint8_t reftable_block_type(const struct reftable_block *b);\n \n /* Decodes the first key in the block */\n-int block_reader_first_key(const struct block_reader *br, struct reftable_buf *key);\n+int reftable_block_first_key(const struct reftable_block *b, struct reftable_buf *key);\n \n /* Iterate over entries in a block */\n struct block_iter {\n@@ -125,10 +130,10 @@ struct block_iter {\n }\n \n /* Position `it` at start of the block */\n-void block_iter_seek_start(struct block_iter *it, const struct block_reader *br);\n+void block_iter_seek_start(struct block_iter *it, const struct reftable_block *block);\n \n /* Position `it` to the `want` key in the block */\n-int block_iter_seek_key(struct block_iter *it, const struct block_reader *br,\n+int block_iter_seek_key(struct block_iter *it, const struct reftable_block *block,\n \t\t\tstruct reftable_buf *want);\n \n /* return < 0 for error, 0 for OK, > 0 for EOF. */\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex c00ffc88577..9ce675510c9 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -114,7 +114,7 @@ static void indexed_table_ref_iter_close(void *p)\n {\n \tstruct indexed_table_ref_iter *it = p;\n \tblock_iter_close(&it->cur);\n-\tblock_source_release_data(&it->block_reader.block_data);\n+\tblock_source_release_data(&it->block.block_data);\n \treftable_free(it->offsets);\n \treftable_buf_release(&it->oid);\n }\n@@ -128,11 +128,10 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n \t\treturn 1;\n \t}\n \n-\tblock_source_release_data(&it->block_reader.block_data);\n+\tblock_source_release_data(&it->block.block_data);\n \n \toff = it->offsets[it->offset_idx++];\n-\terr = table_init_block_reader(it->table, &it->block_reader, off,\n-\t\t\t\t      BLOCK_TYPE_REF);\n+\terr = table_init_block(it->table, &it->block, off, BLOCK_TYPE_REF);\n \tif (err < 0) {\n \t\treturn err;\n \t}\n@@ -140,7 +139,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n \t\t/* indexed block does not exist. */\n \t\treturn REFTABLE_FORMAT_ERROR;\n \t}\n-\tblock_iter_seek_start(&it->cur, &it->block_reader);\n+\tblock_iter_seek_start(&it->cur, &it->block);\n \treturn 0;\n }\n \ndiff --git a/reftable/iter.h b/reftable/iter.h\nindex add5c9472fc..cc920970a57 100644\n--- a/reftable/iter.h\n+++ b/reftable/iter.h\n@@ -68,7 +68,7 @@ struct indexed_table_ref_iter {\n \t/* Points to the next offset to read. */\n \tint offset_idx;\n \tint offset_len;\n-\tstruct block_reader block_reader;\n+\tstruct reftable_block block;\n \tstruct block_iter cur;\n \tint is_finished;\n };\ndiff --git a/reftable/table.c b/reftable/table.c\nindex c86b1d4c649..ef1f33c92fc 100644\n--- a/reftable/table.c\n+++ b/reftable/table.c\n@@ -131,7 +131,7 @@ struct table_iter {\n \tstruct reftable_table *table;\n \tuint8_t typ;\n \tuint64_t block_off;\n-\tstruct block_reader br;\n+\tstruct reftable_block block;\n \tstruct block_iter bi;\n \tint is_finished;\n };\n@@ -159,12 +159,12 @@ static int table_iter_next_in_block(struct table_iter *ti,\n \n static void table_iter_block_done(struct table_iter *ti)\n {\n-\tblock_reader_release(&ti->br);\n+\treftable_block_release(&ti->block);\n \tblock_iter_reset(&ti->bi);\n }\n \n-int table_init_block_reader(struct reftable_table *t, struct block_reader *br,\n-\t\t\t    uint64_t next_off, uint8_t want_typ)\n+int table_init_block(struct reftable_table *t, struct reftable_block *block,\n+\t\t     uint64_t next_off, uint8_t want_typ)\n {\n \tuint32_t header_off = next_off ? 0 : header_size(t->version);\n \tint err;\n@@ -172,19 +172,19 @@ int table_init_block_reader(struct reftable_table *t, struct block_reader *br,\n \tif (next_off >= t->size)\n \t\treturn 1;\n \n-\terr = block_reader_init(br, &t->source, next_off, header_off,\n-\t\t\t\tt->block_size, hash_size(t->hash_id));\n+\terr = reftable_block_init(block, &t->source, next_off, header_off,\n+\t\t\t\t  t->block_size, hash_size(t->hash_id));\n \tif (err < 0)\n \t\tgoto done;\n \n-\tif (want_typ != BLOCK_TYPE_ANY && br->block_type != want_typ) {\n+\tif (want_typ != BLOCK_TYPE_ANY && block->block_type != want_typ) {\n \t\terr = 1;\n \t\tgoto done;\n \t}\n \n done:\n \tif (err)\n-\t\tblock_reader_release(br);\n+\t\treftable_block_release(block);\n \treturn err;\n }\n \n@@ -197,10 +197,10 @@ static void table_iter_close(struct table_iter *ti)\n \n static int table_iter_next_block(struct table_iter *ti)\n {\n-\tuint64_t next_block_off = ti->block_off + ti->br.full_block_size;\n+\tuint64_t next_block_off = ti->block_off + ti->block.full_block_size;\n \tint err;\n \n-\terr = table_init_block_reader(ti->table, &ti->br, next_block_off, ti->typ);\n+\terr = table_init_block(ti->table, &ti->block, next_block_off, ti->typ);\n \tif (err > 0)\n \t\tti->is_finished = 1;\n \tif (err)\n@@ -208,7 +208,7 @@ static int table_iter_next_block(struct table_iter *ti)\n \n \tti->block_off = next_block_off;\n \tti->is_finished = 0;\n-\tblock_iter_seek_start(&ti->bi, &ti->br);\n+\tblock_iter_seek_start(&ti->bi, &ti->block);\n \n \treturn 0;\n }\n@@ -250,13 +250,13 @@ static int table_iter_seek_to(struct table_iter *ti, uint64_t off, uint8_t typ)\n {\n \tint err;\n \n-\terr = table_init_block_reader(ti->table, &ti->br, off, typ);\n+\terr = table_init_block(ti->table, &ti->block, off, typ);\n \tif (err != 0)\n \t\treturn err;\n \n-\tti->typ = block_reader_type(&ti->br);\n+\tti->typ = reftable_block_type(&ti->block);\n \tti->block_off = off;\n-\tblock_iter_seek_start(&ti->bi, &ti->br);\n+\tblock_iter_seek_start(&ti->bi, &ti->block);\n \tti->is_finished = 0;\n \treturn 0;\n }\n@@ -320,10 +320,10 @@ static int table_iter_seek_linear(struct table_iter *ti,\n \t\t * as we have more than three blocks we would have an index, so\n \t\t * we would not do a linear search there anymore.\n \t\t */\n-\t\tmemset(&next.br.block_data, 0, sizeof(next.br.block_data));\n-\t\tnext.br.zstream = NULL;\n-\t\tnext.br.uncompressed_data = NULL;\n-\t\tnext.br.uncompressed_cap = 0;\n+\t\tmemset(&next.block.block_data, 0, sizeof(next.block.block_data));\n+\t\tnext.block.zstream = NULL;\n+\t\tnext.block.uncompressed_data = NULL;\n+\t\tnext.block.uncompressed_cap = 0;\n \n \t\terr = table_iter_next_block(&next);\n \t\tif (err < 0)\n@@ -331,7 +331,7 @@ static int table_iter_seek_linear(struct table_iter *ti,\n \t\tif (err > 0)\n \t\t\tbreak;\n \n-\t\terr = block_reader_first_key(&next.br, &got_key);\n+\t\terr = reftable_block_first_key(&next.block, &got_key);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n@@ -349,7 +349,7 @@ static int table_iter_seek_linear(struct table_iter *ti,\n \t * the wanted key inside of it. If the block does not contain our key\n \t * we know that the corresponding record does not exist.\n \t */\n-\terr = block_iter_seek_key(&ti->bi, &ti->br, &want_key);\n+\terr = block_iter_seek_key(&ti->bi, &ti->block, &want_key);\n \tif (err < 0)\n \t\tgoto done;\n \terr = 0;\n@@ -417,7 +417,7 @@ static int table_iter_seek_indexed(struct table_iter *ti,\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = block_iter_seek_key(&ti->bi, &ti->br, &want_index.u.idx.last_key);\n+\t\terr = block_iter_seek_key(&ti->bi, &ti->block, &want_index.u.idx.last_key);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n@@ -785,8 +785,8 @@ int reftable_table_print_blocks(const char *tablename)\n \t\tprintf(\"%s:\\n\", sections[i].name);\n \n \t\twhile (1) {\n-\t\t\tprintf(\"  - length: %u\\n\", ti.br.restart_off);\n-\t\t\tprintf(\"    restarts: %u\\n\", ti.br.restart_count);\n+\t\t\tprintf(\"  - length: %u\\n\", ti.block.restart_off);\n+\t\t\tprintf(\"    restarts: %u\\n\", ti.block.restart_count);\n \n \t\t\terr = table_iter_next_block(&ti);\n \t\t\tif (err < 0)\ndiff --git a/reftable/table.h b/reftable/table.h\nindex 8d8dd2b413d..e15d58d8e9e 100644\n--- a/reftable/table.h\n+++ b/reftable/table.h\n@@ -53,8 +53,10 @@ int table_init_iter(struct reftable_table *t,\n \t\t    struct reftable_iterator *it,\n \t\t    uint8_t typ);\n \n-/* initialize a block reader to read from `t` */\n-int table_init_block_reader(struct reftable_table *t, struct block_reader *br,\n-\t\t\t    uint64_t next_off, uint8_t want_typ);\n+/*\n+ * Initialize a block by reading from the given table and offset.\n+ */\n+int table_init_block(struct reftable_table *t, struct reftable_block *block,\n+\t\t     uint64_t next_off, uint8_t want_typ);\n \n #endif\ndiff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c\nindex dbec6a5d0d0..e36ed7ac576 100644\n--- a/t/unit-tests/t-reftable-block.c\n+++ b/t/unit-tests/t-reftable-block.c\n@@ -28,16 +28,16 @@ static void t_ref_block_read_write(void)\n \t};\n \tsize_t i = 0;\n \tint ret;\n-\tstruct block_reader br = { 0 };\n+\tstruct reftable_block block = { 0 };\n \tstruct block_iter it = BLOCK_ITER_INIT;\n \tstruct reftable_buf want = REFTABLE_BUF_INIT;\n-\tstruct reftable_buf block = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf block_data = REFTABLE_BUF_INIT;\n \n-\tREFTABLE_CALLOC_ARRAY(block.buf, block_size);\n-\tcheck(block.buf != NULL);\n-\tblock.len = block_size;\n+\tREFTABLE_CALLOC_ARRAY(block_data.buf, block_size);\n+\tcheck(block_data.buf != NULL);\n+\tblock_data.len = block_size;\n \n-\tret = block_writer_init(&bw, BLOCK_TYPE_REF, (uint8_t *) block.buf, block_size,\n+\tret = block_writer_init(&bw, BLOCK_TYPE_REF, (uint8_t *) block_data.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -63,10 +63,10 @@ static void t_ref_block_read_write(void)\n \n \tblock_writer_release(&bw);\n \n-\tblock_source_from_buf(&source ,&block);\n-\tblock_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n+\tblock_source_from_buf(&source ,&block_data);\n+\treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n-\tblock_iter_seek_start(&it, &br);\n+\tblock_iter_seek_start(&it, &block);\n \n \tfor (i = 0; ; i++) {\n \t\tret = block_iter_next(&it, &rec);\n@@ -82,7 +82,7 @@ static void t_ref_block_read_write(void)\n \t\tblock_iter_reset(&it);\n \t\treftable_record_key(&recs[i], &want);\n \n-\t\tret = block_iter_seek_key(&it, &br, &want);\n+\t\tret = block_iter_seek_key(&it, &block, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -91,7 +91,7 @@ static void t_ref_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1));\n \n \t\twant.len--;\n-\t\tret = block_iter_seek_key(&it, &br, &want);\n+\t\tret = block_iter_seek_key(&it, &block, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -99,11 +99,11 @@ static void t_ref_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[10 * (i / 10)], &rec, REFTABLE_HASH_SIZE_SHA1));\n \t}\n \n-\tblock_reader_release(&br);\n+\treftable_block_release(&block);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n \treftable_buf_release(&want);\n-\treftable_buf_release(&block);\n+\treftable_buf_release(&block_data);\n \tfor (i = 0; i < N; i++)\n \t\treftable_record_release(&recs[i]);\n }\n@@ -123,16 +123,16 @@ static void t_log_block_read_write(void)\n \t};\n \tsize_t i = 0;\n \tint ret;\n-\tstruct block_reader br = { 0 };\n+\tstruct reftable_block block = { 0 };\n \tstruct block_iter it = BLOCK_ITER_INIT;\n \tstruct reftable_buf want = REFTABLE_BUF_INIT;\n-\tstruct reftable_buf block = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf block_data = REFTABLE_BUF_INIT;\n \n-\tREFTABLE_CALLOC_ARRAY(block.buf, block_size);\n-\tcheck(block.buf != NULL);\n-\tblock.len = block_size;\n+\tREFTABLE_CALLOC_ARRAY(block_data.buf, block_size);\n+\tcheck(block_data.buf != NULL);\n+\tblock_data.len = block_size;\n \n-\tret = block_writer_init(&bw, BLOCK_TYPE_LOG, (uint8_t *) block.buf, block_size,\n+\tret = block_writer_init(&bw, BLOCK_TYPE_LOG, (uint8_t *) block_data.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -153,10 +153,10 @@ static void t_log_block_read_write(void)\n \n \tblock_writer_release(&bw);\n \n-\tblock_source_from_buf(&source, &block);\n-\tblock_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n+\tblock_source_from_buf(&source, &block_data);\n+\treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n-\tblock_iter_seek_start(&it, &br);\n+\tblock_iter_seek_start(&it, &block);\n \n \tfor (i = 0; ; i++) {\n \t\tret = block_iter_next(&it, &rec);\n@@ -173,7 +173,7 @@ static void t_log_block_read_write(void)\n \t\treftable_buf_reset(&want);\n \t\tcheck(!reftable_buf_addstr(&want, recs[i].u.log.refname));\n \n-\t\tret = block_iter_seek_key(&it, &br, &want);\n+\t\tret = block_iter_seek_key(&it, &block, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -182,7 +182,7 @@ static void t_log_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1));\n \n \t\twant.len--;\n-\t\tret = block_iter_seek_key(&it, &br, &want);\n+\t\tret = block_iter_seek_key(&it, &block, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -190,11 +190,11 @@ static void t_log_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[10 * (i / 10)], &rec, REFTABLE_HASH_SIZE_SHA1));\n \t}\n \n-\tblock_reader_release(&br);\n+\treftable_block_release(&block);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n \treftable_buf_release(&want);\n-\treftable_buf_release(&block);\n+\treftable_buf_release(&block_data);\n \tfor (i = 0; i < N; i++)\n \t\treftable_record_release(&recs[i]);\n }\n@@ -214,16 +214,16 @@ static void t_obj_block_read_write(void)\n \t};\n \tsize_t i = 0;\n \tint ret;\n-\tstruct block_reader br = { 0 };\n+\tstruct reftable_block block = { 0 };\n \tstruct block_iter it = BLOCK_ITER_INIT;\n \tstruct reftable_buf want = REFTABLE_BUF_INIT;\n-\tstruct reftable_buf block = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf block_data = REFTABLE_BUF_INIT;\n \n-\tREFTABLE_CALLOC_ARRAY(block.buf, block_size);\n-\tcheck(block.buf != NULL);\n-\tblock.len = block_size;\n+\tREFTABLE_CALLOC_ARRAY(block_data.buf, block_size);\n+\tcheck(block_data.buf != NULL);\n+\tblock_data.len = block_size;\n \n-\tret = block_writer_init(&bw, BLOCK_TYPE_OBJ, (uint8_t *) block.buf, block_size,\n+\tret = block_writer_init(&bw, BLOCK_TYPE_OBJ, (uint8_t *) block_data.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -246,10 +246,10 @@ static void t_obj_block_read_write(void)\n \n \tblock_writer_release(&bw);\n \n-\tblock_source_from_buf(&source, &block);\n-\tblock_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n+\tblock_source_from_buf(&source, &block_data);\n+\treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n-\tblock_iter_seek_start(&it, &br);\n+\tblock_iter_seek_start(&it, &block);\n \n \tfor (i = 0; ; i++) {\n \t\tret = block_iter_next(&it, &rec);\n@@ -265,7 +265,7 @@ static void t_obj_block_read_write(void)\n \t\tblock_iter_reset(&it);\n \t\treftable_record_key(&recs[i], &want);\n \n-\t\tret = block_iter_seek_key(&it, &br, &want);\n+\t\tret = block_iter_seek_key(&it, &block, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -274,11 +274,11 @@ static void t_obj_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1));\n \t}\n \n-\tblock_reader_release(&br);\n+\treftable_block_release(&block);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n \treftable_buf_release(&want);\n-\treftable_buf_release(&block);\n+\treftable_buf_release(&block_data);\n \tfor (i = 0; i < N; i++)\n \t\treftable_record_release(&recs[i]);\n }\n@@ -299,16 +299,16 @@ static void t_index_block_read_write(void)\n \t};\n \tsize_t i = 0;\n \tint ret;\n-\tstruct block_reader br = { 0 };\n+\tstruct reftable_block block = { 0 };\n \tstruct block_iter it = BLOCK_ITER_INIT;\n \tstruct reftable_buf want = REFTABLE_BUF_INIT;\n-\tstruct reftable_buf block = REFTABLE_BUF_INIT;\n+\tstruct reftable_buf block_data = REFTABLE_BUF_INIT;\n \n-\tREFTABLE_CALLOC_ARRAY(block.buf, block_size);\n-\tcheck(block.buf != NULL);\n-\tblock.len = block_size;\n+\tREFTABLE_CALLOC_ARRAY(block_data.buf, block_size);\n+\tcheck(block_data.buf != NULL);\n+\tblock_data.len = block_size;\n \n-\tret = block_writer_init(&bw, BLOCK_TYPE_INDEX, (uint8_t *) block.buf, block_size,\n+\tret = block_writer_init(&bw, BLOCK_TYPE_INDEX, (uint8_t *) block_data.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -331,10 +331,10 @@ static void t_index_block_read_write(void)\n \n \tblock_writer_release(&bw);\n \n-\tblock_source_from_buf(&source, &block);\n-\tblock_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n+\tblock_source_from_buf(&source, &block_data);\n+\treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n-\tblock_iter_seek_start(&it, &br);\n+\tblock_iter_seek_start(&it, &block);\n \n \tfor (i = 0; ; i++) {\n \t\tret = block_iter_next(&it, &rec);\n@@ -350,7 +350,7 @@ static void t_index_block_read_write(void)\n \t\tblock_iter_reset(&it);\n \t\treftable_record_key(&recs[i], &want);\n \n-\t\tret = block_iter_seek_key(&it, &br, &want);\n+\t\tret = block_iter_seek_key(&it, &block, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -359,7 +359,7 @@ static void t_index_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1));\n \n \t\twant.len--;\n-\t\tret = block_iter_seek_key(&it, &br, &want);\n+\t\tret = block_iter_seek_key(&it, &block, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -367,11 +367,11 @@ static void t_index_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[10 * (i / 10)], &rec, REFTABLE_HASH_SIZE_SHA1));\n \t}\n \n-\tblock_reader_release(&br);\n+\treftable_block_release(&block);\n \tblock_iter_close(&it);\n \treftable_record_release(&rec);\n \treftable_buf_release(&want);\n-\treftable_buf_release(&block);\n+\treftable_buf_release(&block_data);\n \tfor (i = 0; i < N; i++)\n \t\treftable_record_release(&recs[i]);\n }\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515771","messageId":"20250407-pks-reftable-polishing-v2-8-316c4ff10571@pks.im","threadId":"63222","inReplyTo":"20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im","subject":"[PATCH v2 08/16] git-zlib: use `struct z_stream_s` instead of typedef","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T13:16:20Z","receivedAt":"2025-04-07T13:16:30Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Throughout the Git codebase we're using the typedeffed version of\n`z_stream`, which maps to `struct z_stream_s`. By using a typedef\ninstead of the struct it becomes somewhat harder to predeclare the\nsymbol so that headers depending on the struct can do so without having\nto pull in \"zlib-compat.h\".\n\nWe don't yet have users that would really care about this: the only\nusers that declare `z_stream` as a pointer are in \"reftable/block.h\",\nwhich is a header that is internal to the reftable library. But in the\nnext step we're going to expose the `struct reftable_block` publicly,\nand that struct does contain a pointer to `z_stream`. And as the public\nheader shouldn't depend on \"reftable/system.h\", which is an internal\nimplementation detail, we won't have the typedef for `z_stream` readily\navailable.\n\nPrepare for this change by using `struct z_stream_s` throughout our code\nbase. In case zlib-ng is used we use a define to map from `z_stream_s`\nto `zng_stream_s`.\n\nDrop the pre-declaration of `struct z_stream` while at it. This struct\ndoes not exist in the first place, and the declaration wasn't needed\nbecause \"reftable/block.h\" already includes \"reftable/basics.h\" which\ntransitively includes \"reftable/system.h\" and thus \"git-zlib.h\".\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n Documentation/howto/recover-corrupted-object-harder.adoc | 4 ++--\n compat/zlib-compat.h                                     | 4 ++--\n git-zlib.h                                               | 2 +-\n reftable/block.h                                         | 6 ++----\n 4 files changed, 7 insertions(+), 9 deletions(-)\n\ndiff --git a/Documentation/howto/recover-corrupted-object-harder.adoc b/Documentation/howto/recover-corrupted-object-harder.adoc\nindex 5efb4fe81ff..86a1ba75cf9 100644\n--- a/Documentation/howto/recover-corrupted-object-harder.adoc\n+++ b/Documentation/howto/recover-corrupted-object-harder.adoc\n@@ -125,7 +125,7 @@ static int try_zlib(unsigned char *buf, int len)\n {\n \t/* make this absurdly large so we don't have to loop */\n \tstatic unsigned char out[1024*1024];\n-\tz_stream z;\n+\tstruct z_stream_s z;\n \tint ret;\n \n \tmemset(&z, 0, sizeof(z));\n@@ -278,7 +278,7 @@ int main(int argc, char **argv)\n \tstatic unsigned char buf[25 * 1024 * 1024];\n \tstatic unsigned char out[25 * 1024 * 1024];\n \tint len;\n-\tz_stream z;\n+\tstruct z_stream_s z;\n \tint ret;\n \n \tlen = read(0, buf, sizeof(buf));\ndiff --git a/compat/zlib-compat.h b/compat/zlib-compat.h\nindex 0c60e3af331..ac082766229 100644\n--- a/compat/zlib-compat.h\n+++ b/compat/zlib-compat.h\n@@ -4,8 +4,8 @@\n #ifdef HAVE_ZLIB_NG\n # include <zlib-ng.h>\n \n-# define z_stream zng_stream\n-#define gz_header_s zng_gz_header_s\n+# define z_stream_s zng_stream_s\n+# define gz_header_s zng_gz_header_s\n \n # define crc32(crc, buf, len) zng_crc32(crc, buf, len)\n \ndiff --git a/git-zlib.h b/git-zlib.h\nindex 1e8d9aabcb4..0e66fefa8c9 100644\n--- a/git-zlib.h\n+++ b/git-zlib.h\n@@ -4,7 +4,7 @@\n #include \"compat/zlib-compat.h\"\n \n typedef struct git_zstream {\n-\tz_stream z;\n+\tstruct z_stream_s z;\n \tunsigned long avail_in;\n \tunsigned long avail_out;\n \tunsigned long total_in;\ndiff --git a/reftable/block.h b/reftable/block.h\nindex 3957aee4293..422e2f872c5 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -18,7 +18,7 @@\n  * allocation overhead.\n  */\n struct block_writer {\n-\tz_stream *zstream;\n+\tstruct z_stream_s *zstream;\n \tunsigned char *compressed;\n \tsize_t compressed_cap;\n \n@@ -62,8 +62,6 @@ int block_writer_finish(struct block_writer *w);\n /* clears out internally allocated block_writer members. */\n void block_writer_release(struct block_writer *bw);\n \n-struct z_stream;\n-\n /*\n  * A block part of a reftable. Contains records as well as some metadata\n  * describing them.\n@@ -78,7 +76,7 @@ struct reftable_block {\n \tuint32_t hash_size;\n \n \t/* Uncompressed data for log entries. */\n-\tz_stream *zstream;\n+\tstruct z_stream_s *zstream;\n \tunsigned char *uncompressed_data;\n \tsize_t uncompressed_cap;\n \n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515772","messageId":"20250407-pks-reftable-polishing-v2-9-316c4ff10571@pks.im","threadId":"63222","inReplyTo":"20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im","subject":"[PATCH v2 09/16] reftable/block: create public interface for reading blocks","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T13:16:21Z","receivedAt":"2025-04-07T13:16:31Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"While users of the reftable library wouldn't generally require access to\nindividual blocks in a reftable table, there are valid usecases where\none may require low-level access to them. One such upcoming usecase in\nthe Git codebase is to implement consistency checks for the reftable\nlibrary where we want to verify each block individually.\n\nCreate a public interface for reading blocks. The interface isn't yet\ncomplete and lacks e.g. a way to read individual records from a block.\nSuch missing functionality will be backfilled in subsequent commits.\n\nNote that this change also requires us to expose `reftable_buf`, which\nis used by the `reftable_block_first_key()` function.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/basics.h          |  7 -----\n reftable/block.h           | 48 +-------------------------------\n reftable/reftable-basics.h |  8 ++++++\n reftable/reftable-block.h  | 69 ++++++++++++++++++++++++++++++++++++++++++++++\n 4 files changed, 78 insertions(+), 54 deletions(-)\n\ndiff --git a/reftable/basics.h b/reftable/basics.h\nindex 96a2f0d3829..d8888c12629 100644\n--- a/reftable/basics.h\n+++ b/reftable/basics.h\n@@ -18,13 +18,6 @@\n \n #define REFTABLE_UNUSED __attribute__((__unused__))\n \n-struct reftable_buf {\n-\tsize_t alloc;\n-\tsize_t len;\n-\tchar *buf;\n-};\n-#define REFTABLE_BUF_INIT { 0 }\n-\n /*\n  * Initialize the buffer such that it is ready for use. This is equivalent to\n  * using REFTABLE_BUF_INIT for stack-allocated variables.\ndiff --git a/reftable/block.h b/reftable/block.h\nindex 422e2f872c5..4f7f29028c4 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -11,6 +11,7 @@\n \n #include \"basics.h\"\n #include \"record.h\"\n+#include \"reftable-block.h\"\n #include \"reftable-blocksource.h\"\n \n /*\n@@ -62,53 +63,6 @@ int block_writer_finish(struct block_writer *w);\n /* clears out internally allocated block_writer members. */\n void block_writer_release(struct block_writer *bw);\n \n-/*\n- * A block part of a reftable. Contains records as well as some metadata\n- * describing them.\n- */\n-struct reftable_block {\n-\t/* offset of the block header; nonzero for the first block in a\n-\t * reftable. */\n-\tuint32_t header_off;\n-\n-\t/* the memory block */\n-\tstruct reftable_block_data block_data;\n-\tuint32_t hash_size;\n-\n-\t/* Uncompressed data for log entries. */\n-\tstruct z_stream_s *zstream;\n-\tunsigned char *uncompressed_data;\n-\tsize_t uncompressed_cap;\n-\n-\t/*\n-\t * Restart point data. Restart points are located after the block's\n-\t * record data.\n-\t */\n-\tuint16_t restart_count;\n-\tuint32_t restart_off;\n-\n-\t/* size of the data in the file. For log blocks, this is the compressed\n-\t * size. */\n-\tuint32_t full_block_size;\n-\tuint8_t block_type;\n-};\n-\n-/*\n- * Initialize a reftable block from the given block source.\n- */\n-int reftable_block_init(struct reftable_block *b,\n-\t\t\tstruct reftable_block_source *source,\n-\t\t\tuint32_t offset, uint32_t header_size,\n-\t\t\tuint32_t table_block_size, uint32_t hash_size);\n-\n-void reftable_block_release(struct reftable_block *b);\n-\n-/* Returns the block type (eg. 'r' for refs) */\n-uint8_t reftable_block_type(const struct reftable_block *b);\n-\n-/* Decodes the first key in the block */\n-int reftable_block_first_key(const struct reftable_block *b, struct reftable_buf *key);\n-\n /* Iterate over entries in a block */\n struct block_iter {\n \t/* offset within the block of the next entry to read. */\ndiff --git a/reftable/reftable-basics.h b/reftable/reftable-basics.h\nindex ed7c7c9ac2c..6d73f19c85b 100644\n--- a/reftable/reftable-basics.h\n+++ b/reftable/reftable-basics.h\n@@ -11,6 +11,14 @@\n \n #include <stddef.h>\n \n+/* A buffer that contains arbitrary byte slices. */\n+struct reftable_buf {\n+\tsize_t alloc;\n+\tsize_t len;\n+\tchar *buf;\n+};\n+#define REFTABLE_BUF_INIT { 0 }\n+\n /*\n  * Hash functions understood by the reftable library. Note that the values are\n  * arbitrary and somewhat random such that we can easily detect cases where the\ndiff --git a/reftable/reftable-block.h b/reftable/reftable-block.h\nnew file mode 100644\nindex 00000000000..13bd68be8c5\n--- /dev/null\n+++ b/reftable/reftable-block.h\n@@ -0,0 +1,69 @@\n+/*\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n+\n+#ifndef REFTABLE_BLOCK_H\n+#define REFTABLE_BLOCK_H\n+\n+#include <stdint.h>\n+\n+#include \"reftable-basics.h\"\n+#include \"reftable-blocksource.h\"\n+\n+struct z_stream_s;\n+\n+/*\n+ * A block part of a reftable. Contains records as well as some metadata\n+ * describing them.\n+ */\n+struct reftable_block {\n+\t/*\n+\t * Offset of the block header; nonzero for the first block in a\n+\t * reftable.\n+\t */\n+\tuint32_t header_off;\n+\n+\t/* The memory block. */\n+\tstruct reftable_block_data block_data;\n+\tuint32_t hash_size;\n+\n+\t/* Uncompressed data for log entries. */\n+\tstruct z_stream_s *zstream;\n+\tunsigned char *uncompressed_data;\n+\tsize_t uncompressed_cap;\n+\n+\t/*\n+\t * Restart point data. Restart points are located after the block's\n+\t * record data.\n+\t */\n+\tuint16_t restart_count;\n+\tuint32_t restart_off;\n+\n+\t/*\n+\t * Size of the data in the file. For log blocks, this is the compressed\n+\t * size.\n+\t */\n+\tuint32_t full_block_size;\n+\tuint8_t block_type;\n+};\n+\n+/* Initialize a reftable block from the given block source. */\n+int reftable_block_init(struct reftable_block *b,\n+\t\t\tstruct reftable_block_source *source,\n+\t\t\tuint32_t offset, uint32_t header_size,\n+\t\t\tuint32_t table_block_size, uint32_t hash_size);\n+\n+/* Release resources allocated by the block. */\n+void reftable_block_release(struct reftable_block *b);\n+\n+/* Returns the block type (eg. 'r' for refs). */\n+uint8_t reftable_block_type(const struct reftable_block *b);\n+\n+/* Decodes the first key in the block. */\n+int reftable_block_first_key(const struct reftable_block *b, struct reftable_buf *key);\n+\n+#endif /* REFTABLE_BLOCK_H */\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515773","messageId":"20250407-pks-reftable-polishing-v2-10-316c4ff10571@pks.im","threadId":"63222","inReplyTo":"20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im","subject":"[PATCH v2 10/16] reftable/block: store block pointer in the block iterator","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T13:16:22Z","receivedAt":"2025-04-07T13:16:33Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The block iterator requires access to a bunch of data from the\nunderlying `reftable_block` that it is iterating over. This data is\nstored by copying over relevant data into a separate set of variables.\nThis has multiple downsides:\n\n  - We require more storage space than necessary. This is more of a\n    theoretical issue as we shouldn't ever have many blocks.\n\n  - We have to perform more bookkeeping, and the variable names are\n    inconsistent across the two data structures. This can lead to some\n    confusion.\n\n  - The lifetime of the block iterator is tied to the block anyway, but\n    we hide that a bit by only storing pointers pointing into the block.\n\nThere isn't really any good reason why we rip out parts of the block\ninstead of storing a pointer to the block itself.\n\nRefactor the code to do so. Despite being simpler, it also allows us to\ndecouple the lifetime of the block iterator from seeking in a subsequent\ncommit.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/block.c | 22 ++++++++--------------\n reftable/block.h |  4 +---\n 2 files changed, 9 insertions(+), 17 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 844da397833..a425dd7abf3 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -381,13 +381,11 @@ static uint32_t block_restart_offset(const struct reftable_block *b, size_t idx)\n \treturn reftable_get_be24(b->block_data.data + b->restart_off + 3 * idx);\n }\n \n-void block_iter_seek_start(struct block_iter *it, const struct reftable_block *b)\n+void block_iter_seek_start(struct block_iter *it, const struct reftable_block *block)\n {\n-\tit->block = b->block_data.data;\n-\tit->block_len = b->restart_off;\n-\tit->hash_size = b->hash_size;\n+\tit->block = block;\n \treftable_buf_reset(&it->last_key);\n-\tit->next_off = b->header_off + 4;\n+\tit->next_off = block->header_off + 4;\n }\n \n struct restart_needle_less_args {\n@@ -435,14 +433,14 @@ static int restart_needle_less(size_t idx, void *_args)\n int block_iter_next(struct block_iter *it, struct reftable_record *rec)\n {\n \tstruct string_view in = {\n-\t\t.buf = (unsigned char *) it->block + it->next_off,\n-\t\t.len = it->block_len - it->next_off,\n+\t\t.buf = (unsigned char *) it->block->block_data.data + it->next_off,\n+\t\t.len = it->block->restart_off - it->next_off,\n \t};\n \tstruct string_view start = in;\n \tuint8_t extra = 0;\n \tint n = 0;\n \n-\tif (it->next_off >= it->block_len)\n+\tif (it->next_off >= it->block->restart_off)\n \t\treturn 1;\n \n \tn = reftable_decode_key(&it->last_key, &extra, in);\n@@ -452,7 +450,7 @@ int block_iter_next(struct block_iter *it, struct reftable_record *rec)\n \t\treturn REFTABLE_FORMAT_ERROR;\n \n \tstring_view_consume(&in, n);\n-\tn = reftable_record_decode(rec, it->last_key, extra, in, it->hash_size,\n+\tn = reftable_record_decode(rec, it->last_key, extra, in, it->block->hash_size,\n \t\t\t\t   &it->scratch);\n \tif (n < 0)\n \t\treturn -1;\n@@ -467,8 +465,6 @@ void block_iter_reset(struct block_iter *it)\n \treftable_buf_reset(&it->last_key);\n \tit->next_off = 0;\n \tit->block = NULL;\n-\tit->block_len = 0;\n-\tit->hash_size = 0;\n }\n \n void block_iter_close(struct block_iter *it)\n@@ -528,9 +524,7 @@ int block_iter_seek_key(struct block_iter *it, const struct reftable_block *bloc\n \t\tit->next_off = block_restart_offset(block, i - 1);\n \telse\n \t\tit->next_off = block->header_off + 4;\n-\tit->block = block->block_data.data;\n-\tit->block_len = block->restart_off;\n-\tit->hash_size = block->hash_size;\n+\tit->block = block;\n \n \terr = reftable_record_init(&rec, reftable_block_type(block));\n \tif (err < 0)\ndiff --git a/reftable/block.h b/reftable/block.h\nindex 4f7f29028c4..268d5a1e005 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -67,9 +67,7 @@ void block_writer_release(struct block_writer *bw);\n struct block_iter {\n \t/* offset within the block of the next entry to read. */\n \tuint32_t next_off;\n-\tconst unsigned char *block;\n-\tsize_t block_len;\n-\tuint32_t hash_size;\n+\tconst struct reftable_block *block;\n \n \t/* key for last entry we read. */\n \tstruct reftable_buf last_key;\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515774","messageId":"20250407-pks-reftable-polishing-v2-11-316c4ff10571@pks.im","threadId":"63222","inReplyTo":"20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im","subject":"[PATCH v2 11/16] reftable/block: make block iterators reseekable","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T13:16:23Z","receivedAt":"2025-04-07T13:16:34Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Refactor the block iterators so that initialization and seeking are\ndifferent from one another. This makes the iterator trivially reseekable\nby storing the pointer to the block at initialization time, which we can\nthen reuse on every seek.\n\nThis refactoring prepares the code for exposing a `reftable_iterator`\ninterface for blocks in a subsequent commit. Callsites are adjusted\naccordingly.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/block.c                | 23 +++++++++++++----------\n reftable/block.h                | 21 ++++++++++++++++-----\n reftable/iter.c                 |  2 +-\n reftable/table.c                | 11 +++++++----\n t/unit-tests/t-reftable-block.c | 26 +++++++++++---------------\n 5 files changed, 48 insertions(+), 35 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex a425dd7abf3..20712be7eee 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -381,11 +381,16 @@ static uint32_t block_restart_offset(const struct reftable_block *b, size_t idx)\n \treturn reftable_get_be24(b->block_data.data + b->restart_off + 3 * idx);\n }\n \n-void block_iter_seek_start(struct block_iter *it, const struct reftable_block *block)\n+void block_iter_init(struct block_iter *it, const struct reftable_block *block)\n {\n \tit->block = block;\n+\tblock_iter_seek_start(it);\n+}\n+\n+void block_iter_seek_start(struct block_iter *it)\n+{\n \treftable_buf_reset(&it->last_key);\n-\tit->next_off = block->header_off + 4;\n+\tit->next_off = it->block->header_off + 4;\n }\n \n struct restart_needle_less_args {\n@@ -473,12 +478,11 @@ void block_iter_close(struct block_iter *it)\n \treftable_buf_release(&it->scratch);\n }\n \n-int block_iter_seek_key(struct block_iter *it, const struct reftable_block *block,\n-\t\t\tstruct reftable_buf *want)\n+int block_iter_seek_key(struct block_iter *it, struct reftable_buf *want)\n {\n \tstruct restart_needle_less_args args = {\n \t\t.needle = *want,\n-\t\t.block = block,\n+\t\t.block = it->block,\n \t};\n \tstruct reftable_record rec;\n \tint err = 0;\n@@ -496,7 +500,7 @@ int block_iter_seek_key(struct block_iter *it, const struct reftable_block *bloc\n \t * restart point. While that works alright, we would end up scanning\n \t * too many record.\n \t */\n-\ti = binsearch(block->restart_count, &restart_needle_less, &args);\n+\ti = binsearch(it->block->restart_count, &restart_needle_less, &args);\n \tif (args.error) {\n \t\terr = REFTABLE_FORMAT_ERROR;\n \t\tgoto done;\n@@ -521,12 +525,11 @@ int block_iter_seek_key(struct block_iter *it, const struct reftable_block *bloc\n \t *     starting from the preceding restart point.\n \t */\n \tif (i > 0)\n-\t\tit->next_off = block_restart_offset(block, i - 1);\n+\t\tit->next_off = block_restart_offset(it->block, i - 1);\n \telse\n-\t\tit->next_off = block->header_off + 4;\n-\tit->block = block;\n+\t\tit->next_off = it->block->header_off + 4;\n \n-\terr = reftable_record_init(&rec, reftable_block_type(block));\n+\terr = reftable_record_init(&rec, reftable_block_type(it->block));\n \tif (err < 0)\n \t\tgoto done;\n \ndiff --git a/reftable/block.h b/reftable/block.h\nindex 268d5a1e005..1bfd44f56aa 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -79,12 +79,23 @@ struct block_iter {\n \t.scratch = REFTABLE_BUF_INIT, \\\n }\n \n-/* Position `it` at start of the block */\n-void block_iter_seek_start(struct block_iter *it, const struct reftable_block *block);\n+/*\n+ * Initialize the block iterator with the given block. The iterator will be\n+ * positioned at the first record contained in the block. The block must remain\n+ * valid until the end of the iterator's lifetime. It is valid to re-initialize\n+ * iterators multiple times.\n+ */\n+void block_iter_init(struct block_iter *it, const struct reftable_block *block);\n+\n+/* Position the initialized iterator at the first record of its block. */\n+void block_iter_seek_start(struct block_iter *it);\n \n-/* Position `it` to the `want` key in the block */\n-int block_iter_seek_key(struct block_iter *it, const struct reftable_block *block,\n-\t\t\tstruct reftable_buf *want);\n+/*\n+ * Position the initialized iterator at the desired record key. It is not an\n+ * error in case the record cannot be found. If so, a subsequent call to\n+ * `block_iter_next()` will indicate that the iterator is exhausted.\n+ */\n+int block_iter_seek_key(struct block_iter *it, struct reftable_buf *want);\n \n /* return < 0 for error, 0 for OK, > 0 for EOF. */\n int block_iter_next(struct block_iter *it, struct reftable_record *rec);\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex 9ce675510c9..a79c90d9ba6 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -139,7 +139,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n \t\t/* indexed block does not exist. */\n \t\treturn REFTABLE_FORMAT_ERROR;\n \t}\n-\tblock_iter_seek_start(&it->cur, &it->block);\n+\tblock_iter_init(&it->cur, &it->block);\n \treturn 0;\n }\n \ndiff --git a/reftable/table.c b/reftable/table.c\nindex ef1f33c92fc..50ffad7edcb 100644\n--- a/reftable/table.c\n+++ b/reftable/table.c\n@@ -208,7 +208,7 @@ static int table_iter_next_block(struct table_iter *ti)\n \n \tti->block_off = next_block_off;\n \tti->is_finished = 0;\n-\tblock_iter_seek_start(&ti->bi, &ti->block);\n+\tblock_iter_init(&ti->bi, &ti->block);\n \n \treturn 0;\n }\n@@ -256,7 +256,7 @@ static int table_iter_seek_to(struct table_iter *ti, uint64_t off, uint8_t typ)\n \n \tti->typ = reftable_block_type(&ti->block);\n \tti->block_off = off;\n-\tblock_iter_seek_start(&ti->bi, &ti->block);\n+\tblock_iter_init(&ti->bi, &ti->block);\n \tti->is_finished = 0;\n \treturn 0;\n }\n@@ -349,7 +349,8 @@ static int table_iter_seek_linear(struct table_iter *ti,\n \t * the wanted key inside of it. If the block does not contain our key\n \t * we know that the corresponding record does not exist.\n \t */\n-\terr = block_iter_seek_key(&ti->bi, &ti->block, &want_key);\n+\tblock_iter_init(&ti->bi, &ti->block);\n+\terr = block_iter_seek_key(&ti->bi, &want_key);\n \tif (err < 0)\n \t\tgoto done;\n \terr = 0;\n@@ -417,7 +418,9 @@ static int table_iter_seek_indexed(struct table_iter *ti,\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = block_iter_seek_key(&ti->bi, &ti->block, &want_index.u.idx.last_key);\n+\t\tblock_iter_init(&ti->bi, &ti->block);\n+\n+\t\terr = block_iter_seek_key(&ti->bi, &want_index.u.idx.last_key);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \ndiff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c\nindex e36ed7ac576..9f91d697f6d 100644\n--- a/t/unit-tests/t-reftable-block.c\n+++ b/t/unit-tests/t-reftable-block.c\n@@ -66,7 +66,7 @@ static void t_ref_block_read_write(void)\n \tblock_source_from_buf(&source ,&block_data);\n \treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n-\tblock_iter_seek_start(&it, &block);\n+\tblock_iter_init(&it, &block);\n \n \tfor (i = 0; ; i++) {\n \t\tret = block_iter_next(&it, &rec);\n@@ -79,10 +79,9 @@ static void t_ref_block_read_write(void)\n \t}\n \n \tfor (i = 0; i < N; i++) {\n-\t\tblock_iter_reset(&it);\n \t\treftable_record_key(&recs[i], &want);\n \n-\t\tret = block_iter_seek_key(&it, &block, &want);\n+\t\tret = block_iter_seek_key(&it, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -91,7 +90,7 @@ static void t_ref_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1));\n \n \t\twant.len--;\n-\t\tret = block_iter_seek_key(&it, &block, &want);\n+\t\tret = block_iter_seek_key(&it, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -156,7 +155,7 @@ static void t_log_block_read_write(void)\n \tblock_source_from_buf(&source, &block_data);\n \treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n-\tblock_iter_seek_start(&it, &block);\n+\tblock_iter_init(&it, &block);\n \n \tfor (i = 0; ; i++) {\n \t\tret = block_iter_next(&it, &rec);\n@@ -169,11 +168,10 @@ static void t_log_block_read_write(void)\n \t}\n \n \tfor (i = 0; i < N; i++) {\n-\t\tblock_iter_reset(&it);\n \t\treftable_buf_reset(&want);\n \t\tcheck(!reftable_buf_addstr(&want, recs[i].u.log.refname));\n \n-\t\tret = block_iter_seek_key(&it, &block, &want);\n+\t\tret = block_iter_seek_key(&it, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -182,7 +180,7 @@ static void t_log_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1));\n \n \t\twant.len--;\n-\t\tret = block_iter_seek_key(&it, &block, &want);\n+\t\tret = block_iter_seek_key(&it, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -249,7 +247,7 @@ static void t_obj_block_read_write(void)\n \tblock_source_from_buf(&source, &block_data);\n \treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n-\tblock_iter_seek_start(&it, &block);\n+\tblock_iter_init(&it, &block);\n \n \tfor (i = 0; ; i++) {\n \t\tret = block_iter_next(&it, &rec);\n@@ -262,10 +260,9 @@ static void t_obj_block_read_write(void)\n \t}\n \n \tfor (i = 0; i < N; i++) {\n-\t\tblock_iter_reset(&it);\n \t\treftable_record_key(&recs[i], &want);\n \n-\t\tret = block_iter_seek_key(&it, &block, &want);\n+\t\tret = block_iter_seek_key(&it, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -334,7 +331,7 @@ static void t_index_block_read_write(void)\n \tblock_source_from_buf(&source, &block_data);\n \treftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1);\n \n-\tblock_iter_seek_start(&it, &block);\n+\tblock_iter_init(&it, &block);\n \n \tfor (i = 0; ; i++) {\n \t\tret = block_iter_next(&it, &rec);\n@@ -347,10 +344,9 @@ static void t_index_block_read_write(void)\n \t}\n \n \tfor (i = 0; i < N; i++) {\n-\t\tblock_iter_reset(&it);\n \t\treftable_record_key(&recs[i], &want);\n \n-\t\tret = block_iter_seek_key(&it, &block, &want);\n+\t\tret = block_iter_seek_key(&it, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n@@ -359,7 +355,7 @@ static void t_index_block_read_write(void)\n \t\tcheck(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1));\n \n \t\twant.len--;\n-\t\tret = block_iter_seek_key(&it, &block, &want);\n+\t\tret = block_iter_seek_key(&it, &want);\n \t\tcheck_int(ret, ==, 0);\n \n \t\tret = block_iter_next(&it, &rec);\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515775","messageId":"20250407-pks-reftable-polishing-v2-12-316c4ff10571@pks.im","threadId":"63222","inReplyTo":"20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im","subject":"[PATCH v2 12/16] reftable/block: expose a generic iterator over reftable records","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T13:16:24Z","receivedAt":"2025-04-07T13:16:35Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Expose a generic iterator over reftable records and expose it via the\npublic interface. Together with an upcoming iterator for reftable blocks\ncontained in a table this will allow users to trivially iterate through\nblocks and their respective records individually.\n\nThis functionality will be used to implement consistency checks for the\nreftable backend, which requires more fine-grained control over how we\nread data.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/block.c                | 56 +++++++++++++++++++++++++++++\n reftable/block.h                |  2 +-\n reftable/reftable-block.h       |  5 +++\n t/unit-tests/t-reftable-block.c | 78 +++++++++++++++++++++++++++++++++++++++++\n 4 files changed, 140 insertions(+), 1 deletion(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 20712be7eee..08e22170d56 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -10,6 +10,7 @@\n \n #include \"blocksource.h\"\n #include \"constants.h\"\n+#include \"iter.h\"\n #include \"record.h\"\n #include \"reftable-error.h\"\n #include \"system.h\"\n@@ -581,6 +582,61 @@ int block_iter_seek_key(struct block_iter *it, struct reftable_buf *want)\n \treturn err;\n }\n \n+static int block_iter_seek_void(void *it, struct reftable_record *want)\n+{\n+\tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n+\tstruct block_iter *bi = it;\n+\tint err;\n+\n+\tif (bi->block->block_type != want->type)\n+\t\treturn REFTABLE_API_ERROR;\n+\n+\terr = reftable_record_key(want, &buf);\n+\tif (err < 0)\n+\t\tgoto out;\n+\n+\terr = block_iter_seek_key(it, &buf);\n+\tif (err < 0)\n+\t\tgoto out;\n+\n+\terr = 0;\n+\n+out:\n+\treftable_buf_release(&buf);\n+\treturn err;\n+}\n+\n+static int block_iter_next_void(void *it, struct reftable_record *rec)\n+{\n+\treturn block_iter_next(it, rec);\n+}\n+\n+static void block_iter_close_void(void *it)\n+{\n+\tblock_iter_close(it);\n+}\n+\n+static struct reftable_iterator_vtable block_iter_vtable = {\n+\t.seek = &block_iter_seek_void,\n+\t.next = &block_iter_next_void,\n+\t.close = &block_iter_close_void,\n+};\n+\n+int reftable_block_init_iterator(const struct reftable_block *b,\n+\t\t\t\t struct reftable_iterator *it)\n+{\n+\tstruct block_iter *bi;\n+\n+\tREFTABLE_CALLOC_ARRAY(bi, 1);\n+\tblock_iter_init(bi, b);\n+\n+\tassert(!it->ops);\n+\tit->iter_arg = bi;\n+\tit->ops = &block_iter_vtable;\n+\n+\treturn 0;\n+}\n+\n void block_writer_release(struct block_writer *bw)\n {\n \tdeflateEnd(bw->zstream);\ndiff --git a/reftable/block.h b/reftable/block.h\nindex 1bfd44f56aa..3d069393922 100644\n--- a/reftable/block.h\n+++ b/reftable/block.h\n@@ -63,7 +63,7 @@ int block_writer_finish(struct block_writer *w);\n /* clears out internally allocated block_writer members. */\n void block_writer_release(struct block_writer *bw);\n \n-/* Iterate over entries in a block */\n+/* Iterator for records contained in a single block. */\n struct block_iter {\n \t/* offset within the block of the next entry to read. */\n \tuint32_t next_off;\ndiff --git a/reftable/reftable-block.h b/reftable/reftable-block.h\nindex 13bd68be8c5..04c3b518c87 100644\n--- a/reftable/reftable-block.h\n+++ b/reftable/reftable-block.h\n@@ -13,6 +13,7 @@\n \n #include \"reftable-basics.h\"\n #include \"reftable-blocksource.h\"\n+#include \"reftable-iterator.h\"\n \n struct z_stream_s;\n \n@@ -60,6 +61,10 @@ int reftable_block_init(struct reftable_block *b,\n /* Release resources allocated by the block. */\n void reftable_block_release(struct reftable_block *b);\n \n+/* Initialize a generic record iterator from the given block. */\n+int reftable_block_init_iterator(const struct reftable_block *b,\n+\t\t\t\t struct reftable_iterator *it);\n+\n /* Returns the block type (eg. 'r' for refs). */\n uint8_t reftable_block_type(const struct reftable_block *b);\n \ndiff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c\nindex 9f91d697f6d..5577a5769ed 100644\n--- a/t/unit-tests/t-reftable-block.c\n+++ b/t/unit-tests/t-reftable-block.c\n@@ -372,12 +372,90 @@ static void t_index_block_read_write(void)\n \t\treftable_record_release(&recs[i]);\n }\n \n+static void t_block_iterator(void)\n+{\n+\tstruct reftable_block_source source = { 0 };\n+\tstruct block_writer writer = {\n+\t\t.last_key = REFTABLE_BUF_INIT,\n+\t};\n+\tstruct reftable_record expected_refs[20];\n+\tstruct reftable_ref_record ref = { 0 };\n+\tstruct reftable_iterator it = { 0 };\n+\tstruct reftable_block block = { 0 };\n+\tstruct reftable_buf data;\n+\tint err;\n+\n+\tdata.len = 1024;\n+\tREFTABLE_CALLOC_ARRAY(data.buf, data.len);\n+\tcheck(data.buf != NULL);\n+\n+\terr = block_writer_init(&writer, BLOCK_TYPE_REF, (uint8_t *) data.buf, data.len,\n+\t\t\t\t0, hash_size(REFTABLE_HASH_SHA1));\n+\tcheck(!err);\n+\n+\tfor (size_t i = 0; i < ARRAY_SIZE(expected_refs); i++) {\n+\t\texpected_refs[i] = (struct reftable_record) {\n+\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t.u.ref = {\n+\t\t\t\t.value_type = REFTABLE_REF_VAL1,\n+\t\t\t\t.refname = xstrfmt(\"refs/heads/branch-%02\"PRIuMAX, (uintmax_t)i),\n+\t\t\t},\n+\t\t};\n+\t\tmemset(expected_refs[i].u.ref.value.val1, i, REFTABLE_HASH_SIZE_SHA1);\n+\n+\t\terr = block_writer_add(&writer, &expected_refs[i]);\n+\t\tcheck_int(err, ==, 0);\n+\t}\n+\n+\terr = block_writer_finish(&writer);\n+\tcheck_int(err, >, 0);\n+\n+\tblock_source_from_buf(&source, &data);\n+\treftable_block_init(&block, &source, 0, 0, data.len, REFTABLE_HASH_SIZE_SHA1);\n+\n+\terr = reftable_block_init_iterator(&block, &it);\n+\tcheck_int(err, ==, 0);\n+\n+\tfor (size_t i = 0; ; i++) {\n+\t\terr = reftable_iterator_next_ref(&it, &ref);\n+\t\tif (err > 0) {\n+\t\t\tcheck_int(i, ==, ARRAY_SIZE(expected_refs));\n+\t\t\tbreak;\n+\t\t}\n+\t\tcheck_int(err, ==, 0);\n+\n+\t\tcheck(reftable_ref_record_equal(&ref, &expected_refs[i].u.ref,\n+\t\t\t\t\t\tREFTABLE_HASH_SIZE_SHA1));\n+\t}\n+\n+\terr = reftable_iterator_seek_ref(&it, \"refs/heads/does-not-exist\");\n+\tcheck_int(err, ==, 0);\n+\terr = reftable_iterator_next_ref(&it, &ref);\n+\tcheck_int(err, ==, 1);\n+\n+\terr = reftable_iterator_seek_ref(&it, \"refs/heads/branch-13\");\n+\tcheck_int(err, ==, 0);\n+\terr = reftable_iterator_next_ref(&it, &ref);\n+\tcheck_int(err, ==, 0);\n+\tcheck(reftable_ref_record_equal(&ref, &expected_refs[13].u.ref,\n+\t\t\t\t\tREFTABLE_HASH_SIZE_SHA1));\n+\n+\tfor (size_t i = 0; i < ARRAY_SIZE(expected_refs); i++)\n+\t\treftable_free(expected_refs[i].u.ref.refname);\n+\treftable_ref_record_release(&ref);\n+\treftable_iterator_destroy(&it);\n+\treftable_block_release(&block);\n+\tblock_writer_release(&writer);\n+\treftable_buf_release(&data);\n+}\n+\n int cmd_main(int argc UNUSED, const char *argv[] UNUSED)\n {\n \tTEST(t_index_block_read_write(), \"read-write operations on index blocks work\");\n \tTEST(t_log_block_read_write(), \"read-write operations on log blocks work\");\n \tTEST(t_obj_block_read_write(), \"read-write operations on obj blocks work\");\n \tTEST(t_ref_block_read_write(), \"read-write operations on ref blocks work\");\n+\tTEST(t_block_iterator(), \"block iterator works\");\n \n \treturn test_done();\n }\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515776","messageId":"20250407-pks-reftable-polishing-v2-13-316c4ff10571@pks.im","threadId":"63222","inReplyTo":"20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im","subject":"[PATCH v2 13/16] reftable/table: add `reftable_table` to the public interface","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T13:16:25Z","receivedAt":"2025-04-07T13:16:36Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The `reftable_table` interface is an internal implementation detail that\ncallers have no access to. Having direct access to this structure is\nimportant though for a subsequent patch series that will implement\nconsistency checks for the reftable backend.\n\nMove the structure into \"reftable-table.h\" so that it part of the public\ninterface.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/reftable-table.h | 32 +++++++++++++++++++++++++++++++-\n reftable/table.h          | 33 ---------------------------------\n 2 files changed, 31 insertions(+), 34 deletions(-)\n\ndiff --git a/reftable/reftable-table.h b/reftable/reftable-table.h\nindex 94379026727..a78db9eea7e 100644\n--- a/reftable/reftable-table.h\n+++ b/reftable/reftable-table.h\n@@ -20,8 +20,38 @@\n  * reftable_merged_table and struct reftable_stack.\n  */\n \n+/* Metadata for a block type. */\n+struct reftable_table_offsets {\n+\tint is_present;\n+\tuint64_t offset;\n+\tuint64_t index_offset;\n+};\n+\n /* The table struct is a handle to an open reftable file. */\n-struct reftable_table;\n+struct reftable_table {\n+\t/* for convenience, associate a name with the instance. */\n+\tchar *name;\n+\tstruct reftable_block_source source;\n+\n+\t/* Size of the file, excluding the footer. */\n+\tuint64_t size;\n+\n+\t/* The hash function used for ref records. */\n+\tenum reftable_hash hash_id;\n+\n+\tuint32_t block_size;\n+\tuint64_t min_update_index;\n+\tuint64_t max_update_index;\n+\t/* Length of the OID keys in the 'o' section */\n+\tint object_id_len;\n+\tint version;\n+\n+\tstruct reftable_table_offsets ref_offsets;\n+\tstruct reftable_table_offsets obj_offsets;\n+\tstruct reftable_table_offsets log_offsets;\n+\n+\tuint64_t refcount;\n+};\n \n /* reftable_table_new opens a reftable for reading. If successful,\n  * returns 0 code and sets pp. The name is used for creating a\ndiff --git a/reftable/table.h b/reftable/table.h\nindex e15d58d8e9e..c54703e6210 100644\n--- a/reftable/table.h\n+++ b/reftable/table.h\n@@ -14,39 +14,6 @@\n #include \"reftable-iterator.h\"\n #include \"reftable-table.h\"\n \n-/* metadata for a block type */\n-struct reftable_table_offsets {\n-\tint is_present;\n-\tuint64_t offset;\n-\tuint64_t index_offset;\n-};\n-\n-/* The state for reading a reftable file. */\n-struct reftable_table {\n-\t/* for convenience, associate a name with the instance. */\n-\tchar *name;\n-\tstruct reftable_block_source source;\n-\n-\t/* Size of the file, excluding the footer. */\n-\tuint64_t size;\n-\n-\t/* The hash function used for ref records. */\n-\tenum reftable_hash hash_id;\n-\n-\tuint32_t block_size;\n-\tuint64_t min_update_index;\n-\tuint64_t max_update_index;\n-\t/* Length of the OID keys in the 'o' section */\n-\tint object_id_len;\n-\tint version;\n-\n-\tstruct reftable_table_offsets ref_offsets;\n-\tstruct reftable_table_offsets obj_offsets;\n-\tstruct reftable_table_offsets log_offsets;\n-\n-\tuint64_t refcount;\n-};\n-\n const char *reftable_table_name(struct reftable_table *t);\n \n int table_init_iter(struct reftable_table *t,\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515777","messageId":"20250407-pks-reftable-polishing-v2-14-316c4ff10571@pks.im","threadId":"63222","inReplyTo":"20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im","subject":"[PATCH v2 14/16] reftable/table: introduce iterator for table blocks","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T13:16:26Z","receivedAt":"2025-04-07T13:16:37Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Introduce a new iterator that allows the caller to iterate through all\nblocks contained in a table. This gives users more fine-grained control\nover how exactly those blocks are being read and exposes information to\ncallers that was previously inaccessible.\n\nThis iterator will be required by a future patch series that adds\nconsistency checks for the reftable backend. In addition to that though\nwe will also reimplement `reftable_table_print_blocks()` on top of this\nnew iterator in a subsequent commit.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/reftable-table.h       |  16 ++++++\n reftable/table.c                |  47 +++++++++++++++++\n t/unit-tests/t-reftable-table.c | 110 ++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 173 insertions(+)\n\ndiff --git a/reftable/reftable-table.h b/reftable/reftable-table.h\nindex a78db9eea7e..4d61ce6cf77 100644\n--- a/reftable/reftable-table.h\n+++ b/reftable/reftable-table.h\n@@ -10,6 +10,7 @@\n #define REFTABLE_TABLE_H\n \n #include \"reftable-iterator.h\"\n+#include \"reftable-block.h\"\n #include \"reftable-blocksource.h\"\n \n /*\n@@ -99,4 +100,19 @@ uint64_t reftable_table_min_update_index(struct reftable_table *t);\n /* print blocks onto stdout for debugging. */\n int reftable_table_print_blocks(const char *tablename);\n \n+/*\n+ * An iterator that iterates through the blocks contained in a given table.\n+ */\n+struct reftable_table_iterator {\n+\tvoid *iter_arg;\n+};\n+\n+int reftable_table_iterator_init(struct reftable_table_iterator *it,\n+\t\t\t\t struct reftable_table *t);\n+\n+void reftable_table_iterator_release(struct reftable_table_iterator *it);\n+\n+int reftable_table_iterator_next(struct reftable_table_iterator *it,\n+\t\t\t\t const struct reftable_block **out);\n+\n #endif\ndiff --git a/reftable/table.c b/reftable/table.c\nindex 50ffad7edcb..a2a0c7a1d2f 100644\n--- a/reftable/table.c\n+++ b/reftable/table.c\n@@ -804,3 +804,50 @@ int reftable_table_print_blocks(const char *tablename)\n \ttable_iter_close(&ti);\n \treturn err;\n }\n+\n+int reftable_table_iterator_init(struct reftable_table_iterator *it,\n+\t\t\t\t struct reftable_table *t)\n+{\n+\tstruct table_iter *ti;\n+\tint err;\n+\n+\tREFTABLE_ALLOC_ARRAY(ti, 1);\n+\tif (!ti)\n+\t\treturn REFTABLE_OUT_OF_MEMORY_ERROR;\n+\n+\terr = table_iter_init(ti, t);\n+\tif (err < 0)\n+\t\tgoto out;\n+\n+\tit->iter_arg = ti;\n+\terr = 0;\n+\n+out:\n+\tif (err < 0)\n+\t\treftable_free(ti);\n+\treturn err;\n+}\n+\n+void reftable_table_iterator_release(struct reftable_table_iterator *it)\n+{\n+\tif (!it->iter_arg)\n+\t\treturn;\n+\ttable_iter_close(it->iter_arg);\n+\treftable_free(it->iter_arg);\n+\tit->iter_arg = NULL;\n+}\n+\n+int reftable_table_iterator_next(struct reftable_table_iterator *it,\n+\t\t\t\t const struct reftable_block **out)\n+{\n+\tstruct table_iter *ti = it->iter_arg;\n+\tint err;\n+\n+\terr = table_iter_next_block(ti);\n+\tif (err)\n+\t\treturn err;\n+\n+\t*out = &ti->block;\n+\n+\treturn 0;\n+}\ndiff --git a/t/unit-tests/t-reftable-table.c b/t/unit-tests/t-reftable-table.c\nindex 77c59dbf46d..ba39cdf9a76 100644\n--- a/t/unit-tests/t-reftable-table.c\n+++ b/t/unit-tests/t-reftable-table.c\n@@ -1,7 +1,10 @@\n #include \"test-lib.h\"\n #include \"lib-reftable.h\"\n #include \"reftable/blocksource.h\"\n+#include \"reftable/constants.h\"\n+#include \"reftable/iter.h\"\n #include \"reftable/table.h\"\n+#include \"strbuf.h\"\n \n static int t_table_seek_once(void)\n {\n@@ -88,9 +91,116 @@ static int t_table_reseek(void)\n \treturn 0;\n }\n \n+static int t_table_block_iterator(void)\n+{\n+\tstruct reftable_block_source source = { 0 };\n+\tstruct reftable_table_iterator it = { 0 };\n+\tstruct reftable_ref_record *records;\n+\tconst struct reftable_block *block;\n+\tstruct reftable_table *table;\n+\tstruct reftable_buf buf = REFTABLE_BUF_INIT;\n+\tstruct {\n+\t\tuint8_t block_type;\n+\t\tuint16_t header_off;\n+\t\tuint16_t restart_count;\n+\t\tuint16_t record_count;\n+\t} expected_blocks[] = {\n+\t\t{\n+\t\t\t.block_type = BLOCK_TYPE_REF,\n+\t\t\t.header_off = 24,\n+\t\t\t.restart_count = 10,\n+\t\t\t.record_count = 158,\n+\t\t},\n+\t\t{\n+\t\t\t.block_type = BLOCK_TYPE_REF,\n+\t\t\t.restart_count = 10,\n+\t\t\t.record_count = 159,\n+\t\t},\n+\t\t{\n+\t\t\t.block_type = BLOCK_TYPE_REF,\n+\t\t\t.restart_count = 10,\n+\t\t\t.record_count = 159,\n+\t\t},\n+\t\t{\n+\t\t\t.block_type = BLOCK_TYPE_REF,\n+\t\t\t.restart_count = 2,\n+\t\t\t.record_count = 24,\n+\t\t},\n+\t\t{\n+\t\t\t.block_type = BLOCK_TYPE_INDEX,\n+\t\t\t.restart_count = 1,\n+\t\t\t.record_count = 4,\n+\t\t},\n+\t\t{\n+\t\t\t.block_type = BLOCK_TYPE_OBJ,\n+\t\t\t.restart_count = 1,\n+\t\t\t.record_count = 1,\n+\t\t},\n+\t};\n+\tconst size_t nrecords = 500;\n+\tint ret;\n+\n+\tREFTABLE_CALLOC_ARRAY(records, nrecords);\n+\tfor (size_t i = 0; i < nrecords; i++) {\n+\t\trecords[i].value_type = REFTABLE_REF_VAL1;\n+\t\trecords[i].refname = xstrfmt(\"refs/heads/branch-%03\"PRIuMAX,\n+\t\t\t\t\t     (uintmax_t) i);\n+\t}\n+\n+\tt_reftable_write_to_buf(&buf, records, nrecords, NULL, 0, NULL);\n+\tblock_source_from_buf(&source, &buf);\n+\n+\tret = reftable_table_new(&table, &source, \"name\");\n+\tcheck(!ret);\n+\n+\tret = reftable_table_iterator_init(&it, table);\n+\tcheck(!ret);\n+\n+\tfor (size_t i = 0; i < ARRAY_SIZE(expected_blocks); i++) {\n+\t\tstruct reftable_iterator record_it = { 0 };\n+\t\tstruct reftable_record record = {\n+\t\t\t.type = expected_blocks[i].block_type,\n+\t\t};\n+\n+\t\tret = reftable_table_iterator_next(&it, &block);\n+\t\tcheck(!ret);\n+\n+\t\tcheck_int(block->block_type, ==, expected_blocks[i].block_type);\n+\t\tcheck_int(block->header_off, ==, expected_blocks[i].header_off);\n+\t\tcheck_int(block->restart_count, ==, expected_blocks[i].restart_count);\n+\n+\t\tret = reftable_block_init_iterator(block, &record_it);\n+\t\tcheck(!ret);\n+\n+\t\tfor (size_t j = 0; ; j++) {\n+\t\t\tret = iterator_next(&record_it, &record);\n+\t\t\tif (ret > 0) {\n+\t\t\t\tcheck_int(j, ==, expected_blocks[i].record_count);\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t\tcheck(!ret);\n+\t\t}\n+\n+\t\treftable_iterator_destroy(&record_it);\n+\t\treftable_record_release(&record);\n+\t}\n+\n+\tret = reftable_table_iterator_next(&it, &block);\n+\tcheck_int(ret, ==, 1);\n+\n+\tfor (size_t i = 0; i < nrecords; i++)\n+\t\treftable_free(records[i].refname);\n+\treftable_table_iterator_release(&it);\n+\treftable_table_decref(table);\n+\treftable_buf_release(&buf);\n+\treftable_free(records);\n+\treturn 0;\n+}\n+\n int cmd_main(int argc UNUSED, const char *argv[] UNUSED)\n {\n \tTEST(t_table_seek_once(), \"table can seek once\");\n \tTEST(t_table_reseek(), \"table can reseek multiple times\");\n+\tTEST(t_table_block_iterator(), \"table can iterate through blocks\");\n \treturn test_done();\n }\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515778","messageId":"20250407-pks-reftable-polishing-v2-15-316c4ff10571@pks.im","threadId":"63222","inReplyTo":"20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im","subject":"[PATCH v2 15/16] reftable/constants: make block types part of the public interface","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T13:16:27Z","receivedAt":"2025-04-07T13:16:39Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Now that reftable blocks can be read individually via the public\ninterface it becomes necessary for callers to be able to distinguish the\ndifferent types of blocks. Expose the relevant constants.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/block.c                 |  4 ++--\n reftable/constants.h             |  6 +-----\n reftable/iter.c                  | 10 +++++-----\n reftable/merged.c                |  4 ++--\n reftable/record.c                | 40 ++++++++++++++++++++--------------------\n reftable/reftable-constants.h    | 18 ++++++++++++++++++\n reftable/stack.c                 |  8 ++++----\n reftable/table.c                 | 40 ++++++++++++++++++++--------------------\n reftable/writer.c                | 22 +++++++++++-----------\n t/unit-tests/t-reftable-block.c  | 22 +++++++++++-----------\n t/unit-tests/t-reftable-merged.c | 12 ++++++------\n t/unit-tests/t-reftable-pq.c     | 10 +++++-----\n t/unit-tests/t-reftable-record.c | 40 ++++++++++++++++++++--------------------\n t/unit-tests/t-reftable-table.c  | 12 ++++++------\n 14 files changed, 131 insertions(+), 117 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 08e22170d56..795815b4762 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -160,7 +160,7 @@ int block_writer_finish(struct block_writer *w)\n \t * Log records are stored zlib-compressed. Note that the compression\n \t * also spans over the restart points we have just written.\n \t */\n-\tif (block_writer_type(w) == BLOCK_TYPE_LOG) {\n+\tif (block_writer_type(w) == REFTABLE_BLOCK_TYPE_LOG) {\n \t\tint block_header_skip = 4 + w->header_off;\n \t\tuLongf src_len = w->next - block_header_skip, compressed_len;\n \t\tint ret;\n@@ -254,7 +254,7 @@ int reftable_block_init(struct reftable_block *block,\n \t\t\tgoto done;\n \t}\n \n-\tif (block_type == BLOCK_TYPE_LOG) {\n+\tif (block_type == REFTABLE_BLOCK_TYPE_LOG) {\n \t\tuint32_t block_header_skip = 4 + header_size;\n \t\tuLong dst_len = block_size - block_header_skip;\n \t\tuLong src_len = block->block_data.len - block_header_skip;\ndiff --git a/reftable/constants.h b/reftable/constants.h\nindex 091728cf033..e3b1aaa5164 100644\n--- a/reftable/constants.h\n+++ b/reftable/constants.h\n@@ -9,11 +9,7 @@\n #ifndef CONSTANTS_H\n #define CONSTANTS_H\n \n-#define BLOCK_TYPE_LOG 'g'\n-#define BLOCK_TYPE_INDEX 'i'\n-#define BLOCK_TYPE_REF 'r'\n-#define BLOCK_TYPE_OBJ 'o'\n-#define BLOCK_TYPE_ANY 0\n+#include \"reftable-constants.h\"\n \n #define MAX_RESTARTS ((1 << 16) - 1)\n #define DEFAULT_BLOCK_SIZE 4096\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex a79c90d9ba6..2ecc52b3366 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -131,7 +131,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n \tblock_source_release_data(&it->block.block_data);\n \n \toff = it->offsets[it->offset_idx++];\n-\terr = table_init_block(it->table, &it->block, off, BLOCK_TYPE_REF);\n+\terr = table_init_block(it->table, &it->block, off, REFTABLE_BLOCK_TYPE_REF);\n \tif (err < 0) {\n \t\treturn err;\n \t}\n@@ -246,7 +246,7 @@ int reftable_iterator_seek_ref(struct reftable_iterator *it,\n \t\t\t       const char *name)\n {\n \tstruct reftable_record want = {\n-\t\t.type = BLOCK_TYPE_REF,\n+\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t\t.u.ref = {\n \t\t\t.refname = (char *)name,\n \t\t},\n@@ -258,7 +258,7 @@ int reftable_iterator_next_ref(struct reftable_iterator *it,\n \t\t\t       struct reftable_ref_record *ref)\n {\n \tstruct reftable_record rec = {\n-\t\t.type = BLOCK_TYPE_REF,\n+\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t\t.u = {\n \t\t\t.ref = *ref\n \t\t},\n@@ -272,7 +272,7 @@ int reftable_iterator_seek_log_at(struct reftable_iterator *it,\n \t\t\t\t  const char *name, uint64_t update_index)\n {\n \tstruct reftable_record want = {\n-\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t\t.u.log = {\n \t\t\t.refname = (char *)name,\n \t\t\t.update_index = update_index,\n@@ -291,7 +291,7 @@ int reftable_iterator_next_log(struct reftable_iterator *it,\n \t\t\t       struct reftable_log_record *log)\n {\n \tstruct reftable_record rec = {\n-\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t\t.u = {\n \t\t\t.log = *log,\n \t\t},\ndiff --git a/reftable/merged.c b/reftable/merged.c\nindex d5b974d660e..733de07454d 100644\n--- a/reftable/merged.c\n+++ b/reftable/merged.c\n@@ -301,13 +301,13 @@ int merged_table_init_iter(struct reftable_merged_table *mt,\n int reftable_merged_table_init_ref_iterator(struct reftable_merged_table *mt,\n \t\t\t\t\t    struct reftable_iterator *it)\n {\n-\treturn merged_table_init_iter(mt, it, BLOCK_TYPE_REF);\n+\treturn merged_table_init_iter(mt, it, REFTABLE_BLOCK_TYPE_REF);\n }\n \n int reftable_merged_table_init_log_iterator(struct reftable_merged_table *mt,\n \t\t\t\t\t    struct reftable_iterator *it)\n {\n-\treturn merged_table_init_iter(mt, it, BLOCK_TYPE_LOG);\n+\treturn merged_table_init_iter(mt, it, REFTABLE_BLOCK_TYPE_LOG);\n }\n \n enum reftable_hash reftable_merged_table_hash_id(struct reftable_merged_table *mt)\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 26cd834d405..ed00a724417 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -69,10 +69,10 @@ int put_var_int(struct string_view *dest, uint64_t value)\n int reftable_is_block_type(uint8_t typ)\n {\n \tswitch (typ) {\n-\tcase BLOCK_TYPE_REF:\n-\tcase BLOCK_TYPE_LOG:\n-\tcase BLOCK_TYPE_OBJ:\n-\tcase BLOCK_TYPE_INDEX:\n+\tcase REFTABLE_BLOCK_TYPE_REF:\n+\tcase REFTABLE_BLOCK_TYPE_LOG:\n+\tcase REFTABLE_BLOCK_TYPE_OBJ:\n+\tcase REFTABLE_BLOCK_TYPE_INDEX:\n \t\treturn 1;\n \t}\n \treturn 0;\n@@ -462,7 +462,7 @@ static int reftable_ref_record_cmp_void(const void *_a, const void *_b)\n \n static struct reftable_record_vtable reftable_ref_record_vtable = {\n \t.key = &reftable_ref_record_key,\n-\t.type = BLOCK_TYPE_REF,\n+\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t.copy_from = &reftable_ref_record_copy_from,\n \t.val_type = &reftable_ref_record_val_type,\n \t.encode = &reftable_ref_record_encode,\n@@ -664,7 +664,7 @@ static int reftable_obj_record_cmp_void(const void *_a, const void *_b)\n \n static struct reftable_record_vtable reftable_obj_record_vtable = {\n \t.key = &reftable_obj_record_key,\n-\t.type = BLOCK_TYPE_OBJ,\n+\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n \t.copy_from = &reftable_obj_record_copy_from,\n \t.val_type = &reftable_obj_record_val_type,\n \t.encode = &reftable_obj_record_encode,\n@@ -1035,7 +1035,7 @@ static int reftable_log_record_is_deletion_void(const void *p)\n \n static struct reftable_record_vtable reftable_log_record_vtable = {\n \t.key = &reftable_log_record_key,\n-\t.type = BLOCK_TYPE_LOG,\n+\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t.copy_from = &reftable_log_record_copy_from,\n \t.val_type = &reftable_log_record_val_type,\n \t.encode = &reftable_log_record_encode,\n@@ -1137,7 +1137,7 @@ static int reftable_index_record_cmp(const void *_a, const void *_b)\n \n static struct reftable_record_vtable reftable_index_record_vtable = {\n \t.key = &reftable_index_record_key,\n-\t.type = BLOCK_TYPE_INDEX,\n+\t.type = REFTABLE_BLOCK_TYPE_INDEX,\n \t.copy_from = &reftable_index_record_copy_from,\n \t.val_type = &reftable_index_record_val_type,\n \t.encode = &reftable_index_record_encode,\n@@ -1280,13 +1280,13 @@ int reftable_log_record_is_deletion(const struct reftable_log_record *log)\n static void *reftable_record_data(struct reftable_record *rec)\n {\n \tswitch (rec->type) {\n-\tcase BLOCK_TYPE_REF:\n+\tcase REFTABLE_BLOCK_TYPE_REF:\n \t\treturn &rec->u.ref;\n-\tcase BLOCK_TYPE_LOG:\n+\tcase REFTABLE_BLOCK_TYPE_LOG:\n \t\treturn &rec->u.log;\n-\tcase BLOCK_TYPE_INDEX:\n+\tcase REFTABLE_BLOCK_TYPE_INDEX:\n \t\treturn &rec->u.idx;\n-\tcase BLOCK_TYPE_OBJ:\n+\tcase REFTABLE_BLOCK_TYPE_OBJ:\n \t\treturn &rec->u.obj;\n \t}\n \tabort();\n@@ -1296,13 +1296,13 @@ static struct reftable_record_vtable *\n reftable_record_vtable(struct reftable_record *rec)\n {\n \tswitch (rec->type) {\n-\tcase BLOCK_TYPE_REF:\n+\tcase REFTABLE_BLOCK_TYPE_REF:\n \t\treturn &reftable_ref_record_vtable;\n-\tcase BLOCK_TYPE_LOG:\n+\tcase REFTABLE_BLOCK_TYPE_LOG:\n \t\treturn &reftable_log_record_vtable;\n-\tcase BLOCK_TYPE_INDEX:\n+\tcase REFTABLE_BLOCK_TYPE_INDEX:\n \t\treturn &reftable_index_record_vtable;\n-\tcase BLOCK_TYPE_OBJ:\n+\tcase REFTABLE_BLOCK_TYPE_OBJ:\n \t\treturn &reftable_obj_record_vtable;\n \t}\n \tabort();\n@@ -1314,11 +1314,11 @@ int reftable_record_init(struct reftable_record *rec, uint8_t typ)\n \trec->type = typ;\n \n \tswitch (typ) {\n-\tcase BLOCK_TYPE_REF:\n-\tcase BLOCK_TYPE_LOG:\n-\tcase BLOCK_TYPE_OBJ:\n+\tcase REFTABLE_BLOCK_TYPE_REF:\n+\tcase REFTABLE_BLOCK_TYPE_LOG:\n+\tcase REFTABLE_BLOCK_TYPE_OBJ:\n \t\treturn 0;\n-\tcase BLOCK_TYPE_INDEX:\n+\tcase REFTABLE_BLOCK_TYPE_INDEX:\n \t\treftable_buf_init(&rec->u.idx.last_key);\n \t\treturn 0;\n \tdefault:\ndiff --git a/reftable/reftable-constants.h b/reftable/reftable-constants.h\nnew file mode 100644\nindex 00000000000..4ae9ba4bacc\n--- /dev/null\n+++ b/reftable/reftable-constants.h\n@@ -0,0 +1,18 @@\n+/*\n+ * Copyright 2020 Google LLC\n+ *\n+ * Use of this source code is governed by a BSD-style\n+ * license that can be found in the LICENSE file or at\n+ * https://developers.google.com/open-source/licenses/bsd\n+ */\n+\n+#ifndef REFTABLE_CONSTANTS_H\n+#define REFTABLE_CONSTANTS_H\n+\n+#define REFTABLE_BLOCK_TYPE_LOG 'g'\n+#define REFTABLE_BLOCK_TYPE_INDEX 'i'\n+#define REFTABLE_BLOCK_TYPE_REF 'r'\n+#define REFTABLE_BLOCK_TYPE_OBJ 'o'\n+#define REFTABLE_BLOCK_TYPE_ANY 0\n+\n+#endif /* REFTABLE_CONSTANTS_H */\ndiff --git a/reftable/stack.c b/reftable/stack.c\nindex cc48e725b14..4caf96aa1d6 100644\n--- a/reftable/stack.c\n+++ b/reftable/stack.c\n@@ -203,14 +203,14 @@ int reftable_stack_init_ref_iterator(struct reftable_stack *st,\n \t\t\t\t      struct reftable_iterator *it)\n {\n \treturn merged_table_init_iter(reftable_stack_merged_table(st),\n-\t\t\t\t      it, BLOCK_TYPE_REF);\n+\t\t\t\t      it, REFTABLE_BLOCK_TYPE_REF);\n }\n \n int reftable_stack_init_log_iterator(struct reftable_stack *st,\n \t\t\t\t     struct reftable_iterator *it)\n {\n \treturn merged_table_init_iter(reftable_stack_merged_table(st),\n-\t\t\t\t      it, BLOCK_TYPE_LOG);\n+\t\t\t\t      it, REFTABLE_BLOCK_TYPE_LOG);\n }\n \n struct reftable_merged_table *\n@@ -1098,7 +1098,7 @@ static int stack_write_compact(struct reftable_stack *st,\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = merged_table_init_iter(mt, &it, BLOCK_TYPE_REF);\n+\terr = merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_REF);\n \tif (err < 0)\n \t\tgoto done;\n \n@@ -1126,7 +1126,7 @@ static int stack_write_compact(struct reftable_stack *st,\n \t}\n \treftable_iterator_destroy(&it);\n \n-\terr = merged_table_init_iter(mt, &it, BLOCK_TYPE_LOG);\n+\terr = merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_LOG);\n \tif (err < 0)\n \t\tgoto done;\n \ndiff --git a/reftable/table.c b/reftable/table.c\nindex a2a0c7a1d2f..1af39423224 100644\n--- a/reftable/table.c\n+++ b/reftable/table.c\n@@ -20,11 +20,11 @@ static struct reftable_table_offsets *\n table_offsets_for(struct reftable_table *t, uint8_t typ)\n {\n \tswitch (typ) {\n-\tcase BLOCK_TYPE_REF:\n+\tcase REFTABLE_BLOCK_TYPE_REF:\n \t\treturn &t->ref_offsets;\n-\tcase BLOCK_TYPE_LOG:\n+\tcase REFTABLE_BLOCK_TYPE_LOG:\n \t\treturn &t->log_offsets;\n-\tcase BLOCK_TYPE_OBJ:\n+\tcase REFTABLE_BLOCK_TYPE_OBJ:\n \t\treturn &t->obj_offsets;\n \t}\n \tabort();\n@@ -112,9 +112,9 @@ static int parse_footer(struct reftable_table *t, uint8_t *footer,\n \t}\n \n \tfirst_block_typ = header[header_size(t->version)];\n-\tt->ref_offsets.is_present = (first_block_typ == BLOCK_TYPE_REF);\n+\tt->ref_offsets.is_present = (first_block_typ == REFTABLE_BLOCK_TYPE_REF);\n \tt->ref_offsets.offset = 0;\n-\tt->log_offsets.is_present = (first_block_typ == BLOCK_TYPE_LOG ||\n+\tt->log_offsets.is_present = (first_block_typ == REFTABLE_BLOCK_TYPE_LOG ||\n \t\t\t\t     t->log_offsets.offset > 0);\n \tt->obj_offsets.is_present = t->obj_offsets.offset > 0;\n \tif (t->obj_offsets.is_present && !t->object_id_len) {\n@@ -150,7 +150,7 @@ static int table_iter_next_in_block(struct table_iter *ti,\n \t\t\t\t    struct reftable_record *rec)\n {\n \tint res = block_iter_next(&ti->bi, rec);\n-\tif (res == 0 && reftable_record_type(rec) == BLOCK_TYPE_REF) {\n+\tif (res == 0 && reftable_record_type(rec) == REFTABLE_BLOCK_TYPE_REF) {\n \t\trec->u.ref.update_index += ti->table->min_update_index;\n \t}\n \n@@ -177,7 +177,7 @@ int table_init_block(struct reftable_table *t, struct reftable_block *block,\n \tif (err < 0)\n \t\tgoto done;\n \n-\tif (want_typ != BLOCK_TYPE_ANY && block->block_type != want_typ) {\n+\tif (want_typ != REFTABLE_BLOCK_TYPE_ANY && block->block_type != want_typ) {\n \t\terr = 1;\n \t\tgoto done;\n \t}\n@@ -270,7 +270,7 @@ static int table_iter_seek_start(struct table_iter *ti, uint8_t typ, int index)\n \t\tif (off == 0) {\n \t\t\treturn 1;\n \t\t}\n-\t\ttyp = BLOCK_TYPE_INDEX;\n+\t\ttyp = REFTABLE_BLOCK_TYPE_INDEX;\n \t}\n \n \treturn table_iter_seek_to(ti, off, typ);\n@@ -366,10 +366,10 @@ static int table_iter_seek_indexed(struct table_iter *ti,\n \t\t\t\t   struct reftable_record *rec)\n {\n \tstruct reftable_record want_index = {\n-\t\t.type = BLOCK_TYPE_INDEX, .u.idx = { .last_key = REFTABLE_BUF_INIT }\n+\t\t.type = REFTABLE_BLOCK_TYPE_INDEX, .u.idx = { .last_key = REFTABLE_BUF_INIT }\n \t};\n \tstruct reftable_record index_result = {\n-\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.type = REFTABLE_BLOCK_TYPE_INDEX,\n \t\t.u.idx = { .last_key = REFTABLE_BUF_INIT },\n \t};\n \tint err;\n@@ -429,7 +429,7 @@ static int table_iter_seek_indexed(struct table_iter *ti,\n \t\t\tbreak;\n \t\t}\n \n-\t\tif (ti->typ != BLOCK_TYPE_INDEX) {\n+\t\tif (ti->typ != REFTABLE_BLOCK_TYPE_INDEX) {\n \t\t\terr = REFTABLE_FORMAT_ERROR;\n \t\t\tgoto done;\n \t\t}\n@@ -517,13 +517,13 @@ int table_init_iter(struct reftable_table *t,\n int reftable_table_init_ref_iterator(struct reftable_table *t,\n \t\t\t\t     struct reftable_iterator *it)\n {\n-\treturn table_init_iter(t, it, BLOCK_TYPE_REF);\n+\treturn table_init_iter(t, it, REFTABLE_BLOCK_TYPE_REF);\n }\n \n int reftable_table_init_log_iterator(struct reftable_table *t,\n \t\t\t\t     struct reftable_iterator *it)\n {\n-\treturn table_init_iter(t, it, BLOCK_TYPE_LOG);\n+\treturn table_init_iter(t, it, REFTABLE_BLOCK_TYPE_LOG);\n }\n \n int reftable_table_new(struct reftable_table **out,\n@@ -625,7 +625,7 @@ static int reftable_table_refs_for_indexed(struct reftable_table *t,\n \t\t\t\t\t   uint8_t *oid)\n {\n \tstruct reftable_record want = {\n-\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n \t\t.u.obj = {\n \t\t\t.hash_prefix = oid,\n \t\t\t.hash_prefix_len = t->object_id_len,\n@@ -633,14 +633,14 @@ static int reftable_table_refs_for_indexed(struct reftable_table *t,\n \t};\n \tstruct reftable_iterator oit = { NULL };\n \tstruct reftable_record got = {\n-\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n \t\t.u.obj = { 0 },\n \t};\n \tint err = 0;\n \tstruct indexed_table_ref_iter *itr = NULL;\n \n \t/* Look through the reverse index. */\n-\terr = table_init_iter(t, &oit, BLOCK_TYPE_OBJ);\n+\terr = table_init_iter(t, &oit, REFTABLE_BLOCK_TYPE_OBJ);\n \tif (err < 0)\n \t\tgoto done;\n \n@@ -692,7 +692,7 @@ static int reftable_table_refs_for_unindexed(struct reftable_table *t,\n \t}\n \n \ttable_iter_init(ti, t);\n-\terr = table_iter_seek_start(ti, BLOCK_TYPE_REF, 0);\n+\terr = table_iter_seek_start(ti, REFTABLE_BLOCK_TYPE_REF, 0);\n \tif (err < 0)\n \t\tgoto out;\n \n@@ -748,15 +748,15 @@ int reftable_table_print_blocks(const char *tablename)\n \t} sections[] = {\n \t\t{\n \t\t\t.name = \"ref\",\n-\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t\t},\n \t\t{\n \t\t\t.name = \"obj\",\n-\t\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n \t\t},\n \t\t{\n \t\t\t.name = \"log\",\n-\t\t\t.type = BLOCK_TYPE_LOG,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t\t},\n \t};\n \tstruct reftable_block_source src = { 0 };\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex f0accfd0c32..0954c29bcf2 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -172,7 +172,7 @@ int reftable_writer_new(struct reftable_writer **out,\n \twp->write_arg = writer_arg;\n \twp->opts = opts;\n \twp->flush = flush_func;\n-\twriter_reinit_block_writer(wp, BLOCK_TYPE_REF);\n+\twriter_reinit_block_writer(wp, REFTABLE_BLOCK_TYPE_REF);\n \n \t*out = wp;\n \n@@ -347,7 +347,7 @@ int reftable_writer_add_ref(struct reftable_writer *w,\n \t\t\t    struct reftable_ref_record *ref)\n {\n \tstruct reftable_record rec = {\n-\t\t.type = BLOCK_TYPE_REF,\n+\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t\t.u = {\n \t\t\t.ref = *ref\n \t\t},\n@@ -411,13 +411,13 @@ static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n \t\t\t\t\t    struct reftable_log_record *log)\n {\n \tstruct reftable_record rec = {\n-\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t\t.u = {\n \t\t\t.log = *log,\n \t\t},\n \t};\n \tif (w->block_writer &&\n-\t    block_writer_type(w->block_writer) == BLOCK_TYPE_REF) {\n+\t    block_writer_type(w->block_writer) == REFTABLE_BLOCK_TYPE_REF) {\n \t\tint err = writer_finish_public_section(w);\n \t\tif (err < 0)\n \t\t\treturn err;\n@@ -537,7 +537,7 @@ static int writer_finish_section(struct reftable_writer *w)\n \n \t\tmax_level++;\n \t\tindex_start = w->next;\n-\t\terr = writer_reinit_block_writer(w, BLOCK_TYPE_INDEX);\n+\t\terr = writer_reinit_block_writer(w, REFTABLE_BLOCK_TYPE_INDEX);\n \t\tif (err < 0)\n \t\t\treturn err;\n \n@@ -549,7 +549,7 @@ static int writer_finish_section(struct reftable_writer *w)\n \t\tw->index_cap = 0;\n \t\tfor (i = 0; i < idx_len; i++) {\n \t\t\tstruct reftable_record rec = {\n-\t\t\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t\t\t.type = REFTABLE_BLOCK_TYPE_INDEX,\n \t\t\t\t.u = {\n \t\t\t\t\t.idx = idx[i],\n \t\t\t\t},\n@@ -614,7 +614,7 @@ static void write_object_record(void *void_arg, void *key)\n \tstruct write_record_arg *arg = void_arg;\n \tstruct obj_index_tree_node *entry = key;\n \tstruct reftable_record\n-\t\trec = { .type = BLOCK_TYPE_OBJ,\n+\t\trec = { .type = REFTABLE_BLOCK_TYPE_OBJ,\n \t\t\t.u.obj = {\n \t\t\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n \t\t\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n@@ -632,7 +632,7 @@ static void write_object_record(void *void_arg, void *key)\n \tif (arg->err < 0)\n \t\tgoto done;\n \n-\targ->err = writer_reinit_block_writer(arg->w, BLOCK_TYPE_OBJ);\n+\targ->err = writer_reinit_block_writer(arg->w, REFTABLE_BLOCK_TYPE_OBJ);\n \tif (arg->err < 0)\n \t\tgoto done;\n \n@@ -670,7 +670,7 @@ static int writer_dump_object_index(struct reftable_writer *w)\n \t\tinfix_walk(w->obj_index_tree, &update_common, &common);\n \tw->stats.object_id_len = common.max + 1;\n \n-\terr = writer_reinit_block_writer(w, BLOCK_TYPE_OBJ);\n+\terr = writer_reinit_block_writer(w, REFTABLE_BLOCK_TYPE_OBJ);\n \tif (err < 0)\n \t\treturn err;\n \n@@ -694,7 +694,7 @@ static int writer_finish_public_section(struct reftable_writer *w)\n \terr = writer_finish_section(w);\n \tif (err < 0)\n \t\treturn err;\n-\tif (typ == BLOCK_TYPE_REF && !w->opts.skip_index_objects &&\n+\tif (typ == REFTABLE_BLOCK_TYPE_REF && !w->opts.skip_index_objects &&\n \t    w->stats.ref_stats.index_blocks > 0) {\n \t\terr = writer_dump_object_index(w);\n \t\tif (err < 0)\n@@ -799,7 +799,7 @@ static int writer_flush_nonempty_block(struct reftable_writer *w)\n \t * By default, all records except for log records are padded to the\n \t * block size.\n \t */\n-\tif (!w->opts.unpadded && typ != BLOCK_TYPE_LOG)\n+\tif (!w->opts.unpadded && typ != REFTABLE_BLOCK_TYPE_LOG)\n \t\tpadding = w->opts.block_size - raw_bytes;\n \n \tbstats = writer_reftable_block_stats(w, typ);\ndiff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c\nindex 5577a5769ed..7dbd93601c7 100644\n--- a/t/unit-tests/t-reftable-block.c\n+++ b/t/unit-tests/t-reftable-block.c\n@@ -24,7 +24,7 @@ static void t_ref_block_read_write(void)\n \t\t.last_key = REFTABLE_BUF_INIT,\n \t};\n \tstruct reftable_record rec = {\n-\t\t.type = BLOCK_TYPE_REF,\n+\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t};\n \tsize_t i = 0;\n \tint ret;\n@@ -37,7 +37,7 @@ static void t_ref_block_read_write(void)\n \tcheck(block_data.buf != NULL);\n \tblock_data.len = block_size;\n \n-\tret = block_writer_init(&bw, BLOCK_TYPE_REF, (uint8_t *) block_data.buf, block_size,\n+\tret = block_writer_init(&bw, REFTABLE_BLOCK_TYPE_REF, (uint8_t *) block_data.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -118,7 +118,7 @@ static void t_log_block_read_write(void)\n \t\t.last_key = REFTABLE_BUF_INIT,\n \t};\n \tstruct reftable_record rec = {\n-\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t};\n \tsize_t i = 0;\n \tint ret;\n@@ -131,7 +131,7 @@ static void t_log_block_read_write(void)\n \tcheck(block_data.buf != NULL);\n \tblock_data.len = block_size;\n \n-\tret = block_writer_init(&bw, BLOCK_TYPE_LOG, (uint8_t *) block_data.buf, block_size,\n+\tret = block_writer_init(&bw, REFTABLE_BLOCK_TYPE_LOG, (uint8_t *) block_data.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -208,7 +208,7 @@ static void t_obj_block_read_write(void)\n \t\t.last_key = REFTABLE_BUF_INIT,\n \t};\n \tstruct reftable_record rec = {\n-\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n \t};\n \tsize_t i = 0;\n \tint ret;\n@@ -221,7 +221,7 @@ static void t_obj_block_read_write(void)\n \tcheck(block_data.buf != NULL);\n \tblock_data.len = block_size;\n \n-\tret = block_writer_init(&bw, BLOCK_TYPE_OBJ, (uint8_t *) block_data.buf, block_size,\n+\tret = block_writer_init(&bw, REFTABLE_BLOCK_TYPE_OBJ, (uint8_t *) block_data.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -291,7 +291,7 @@ static void t_index_block_read_write(void)\n \t\t.last_key = REFTABLE_BUF_INIT,\n \t};\n \tstruct reftable_record rec = {\n-\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.type = REFTABLE_BLOCK_TYPE_INDEX,\n \t\t.u.idx.last_key = REFTABLE_BUF_INIT,\n \t};\n \tsize_t i = 0;\n@@ -305,7 +305,7 @@ static void t_index_block_read_write(void)\n \tcheck(block_data.buf != NULL);\n \tblock_data.len = block_size;\n \n-\tret = block_writer_init(&bw, BLOCK_TYPE_INDEX, (uint8_t *) block_data.buf, block_size,\n+\tret = block_writer_init(&bw, REFTABLE_BLOCK_TYPE_INDEX, (uint8_t *) block_data.buf, block_size,\n \t\t\t\theader_off, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!ret);\n \n@@ -315,7 +315,7 @@ static void t_index_block_read_write(void)\n \t\tsnprintf(buf, sizeof(buf), \"branch%02\"PRIuMAX, (uintmax_t)i);\n \n \t\treftable_buf_init(&recs[i].u.idx.last_key);\n-\t\trecs[i].type = BLOCK_TYPE_INDEX;\n+\t\trecs[i].type = REFTABLE_BLOCK_TYPE_INDEX;\n \t\tcheck(!reftable_buf_addstr(&recs[i].u.idx.last_key, buf));\n \t\trecs[i].u.idx.offset = i;\n \n@@ -389,13 +389,13 @@ static void t_block_iterator(void)\n \tREFTABLE_CALLOC_ARRAY(data.buf, data.len);\n \tcheck(data.buf != NULL);\n \n-\terr = block_writer_init(&writer, BLOCK_TYPE_REF, (uint8_t *) data.buf, data.len,\n+\terr = block_writer_init(&writer, REFTABLE_BLOCK_TYPE_REF, (uint8_t *) data.buf, data.len,\n \t\t\t\t0, hash_size(REFTABLE_HASH_SHA1));\n \tcheck(!err);\n \n \tfor (size_t i = 0; i < ARRAY_SIZE(expected_refs); i++) {\n \t\texpected_refs[i] = (struct reftable_record) {\n-\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t\t\t.u.ref = {\n \t\t\t\t.value_type = REFTABLE_REF_VAL1,\n \t\t\t\t.refname = xstrfmt(\"refs/heads/branch-%02\"PRIuMAX, (uintmax_t)i),\ndiff --git a/t/unit-tests/t-reftable-merged.c b/t/unit-tests/t-reftable-merged.c\nindex fed6beb85c0..18c3251a56a 100644\n--- a/t/unit-tests/t-reftable-merged.c\n+++ b/t/unit-tests/t-reftable-merged.c\n@@ -84,7 +84,7 @@ static void t_merged_single_record(void)\n \tstruct reftable_iterator it = { 0 };\n \tint err;\n \n-\terr = merged_table_init_iter(mt, &it, BLOCK_TYPE_REF);\n+\terr = merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_REF);\n \tcheck(!err);\n \terr = reftable_iterator_seek_ref(&it, \"a\");\n \tcheck(!err);\n@@ -164,7 +164,7 @@ static void t_merged_refs(void)\n \tsize_t cap = 0;\n \tsize_t i;\n \n-\terr = merged_table_init_iter(mt, &it, BLOCK_TYPE_REF);\n+\terr = merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_REF);\n \tcheck(!err);\n \terr = reftable_iterator_seek_ref(&it, \"a\");\n \tcheck(!err);\n@@ -244,7 +244,7 @@ static void t_merged_seek_multiple_times(void)\n \tstruct reftable_merged_table *mt;\n \n \tmt = merged_table_from_records(refs, &sources, &tables, sizes, bufs, 2);\n-\tmerged_table_init_iter(mt, &it, BLOCK_TYPE_REF);\n+\tmerged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_REF);\n \n \tfor (size_t i = 0; i < 5; i++) {\n \t\tint err = reftable_iterator_seek_ref(&it, \"c\");\n@@ -320,7 +320,7 @@ static void t_merged_seek_multiple_times_without_draining(void)\n \tint err;\n \n \tmt = merged_table_from_records(refs, &sources, &tables, sizes, bufs, 2);\n-\tmerged_table_init_iter(mt, &it, BLOCK_TYPE_REF);\n+\tmerged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_REF);\n \n \terr = reftable_iterator_seek_ref(&it, \"b\");\n \tcheck(!err);\n@@ -445,7 +445,7 @@ static void t_merged_logs(void)\n \tsize_t cap = 0;\n \tsize_t i;\n \n-\terr = merged_table_init_iter(mt, &it, BLOCK_TYPE_LOG);\n+\terr = merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_LOG);\n \tcheck(!err);\n \terr = reftable_iterator_seek_log(&it, \"a\");\n \tcheck(!err);\n@@ -469,7 +469,7 @@ static void t_merged_logs(void)\n \t\tcheck(reftable_log_record_equal(want[i], &out[i],\n \t\t\t\t\t\t REFTABLE_HASH_SIZE_SHA1));\n \n-\terr = merged_table_init_iter(mt, &it, BLOCK_TYPE_LOG);\n+\terr = merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_LOG);\n \tcheck(!err);\n \terr = reftable_iterator_seek_log_at(&it, \"a\", 2);\n \tcheck(!err);\ndiff --git a/t/unit-tests/t-reftable-pq.c b/t/unit-tests/t-reftable-pq.c\nindex c128fe8616a..fb5a4eb1877 100644\n--- a/t/unit-tests/t-reftable-pq.c\n+++ b/t/unit-tests/t-reftable-pq.c\n@@ -34,7 +34,7 @@ static void t_pq_record(void)\n \tchar *last = NULL;\n \n \tfor (i = 0; i < N; i++) {\n-\t\tcheck(!reftable_record_init(&recs[i], BLOCK_TYPE_REF));\n+\t\tcheck(!reftable_record_init(&recs[i], REFTABLE_BLOCK_TYPE_REF));\n \t\trecs[i].u.ref.refname = xstrfmt(\"%02\"PRIuMAX, (uintmax_t)i);\n \t}\n \n@@ -57,7 +57,7 @@ static void t_pq_record(void)\n \t\tmerged_iter_pqueue_check(&pq);\n \n \t\tcheck(pq_entry_equal(&top, &e));\n-\t\tcheck(reftable_record_type(e.rec) == BLOCK_TYPE_REF);\n+\t\tcheck(reftable_record_type(e.rec) == REFTABLE_BLOCK_TYPE_REF);\n \t\tif (last)\n \t\t\tcheck_int(strcmp(last, e.rec->u.ref.refname), <, 0);\n \t\tlast = e.rec->u.ref.refname;\n@@ -76,7 +76,7 @@ static void t_pq_index(void)\n \tsize_t N = ARRAY_SIZE(recs), i;\n \n \tfor (i = 0; i < N; i++) {\n-\t\tcheck(!reftable_record_init(&recs[i], BLOCK_TYPE_REF));\n+\t\tcheck(!reftable_record_init(&recs[i], REFTABLE_BLOCK_TYPE_REF));\n \t\trecs[i].u.ref.refname = (char *) \"refs/heads/master\";\n \t}\n \n@@ -100,7 +100,7 @@ static void t_pq_index(void)\n \t\tmerged_iter_pqueue_check(&pq);\n \n \t\tcheck(pq_entry_equal(&top, &e));\n-\t\tcheck(reftable_record_type(e.rec) == BLOCK_TYPE_REF);\n+\t\tcheck(reftable_record_type(e.rec) == REFTABLE_BLOCK_TYPE_REF);\n \t\tcheck_int(e.index, ==, i);\n \t\tif (last)\n \t\t\tcheck_str(last, e.rec->u.ref.refname);\n@@ -117,7 +117,7 @@ static void t_merged_iter_pqueue_top(void)\n \tsize_t N = ARRAY_SIZE(recs), i;\n \n \tfor (i = 0; i < N; i++) {\n-\t\tcheck(!reftable_record_init(&recs[i], BLOCK_TYPE_REF));\n+\t\tcheck(!reftable_record_init(&recs[i], REFTABLE_BLOCK_TYPE_REF));\n \t\trecs[i].u.ref.refname = (char *) \"refs/heads/master\";\n \t}\n \ndiff --git a/t/unit-tests/t-reftable-record.c b/t/unit-tests/t-reftable-record.c\nindex 59549663736..553a0076647 100644\n--- a/t/unit-tests/t-reftable-record.c\n+++ b/t/unit-tests/t-reftable-record.c\n@@ -84,17 +84,17 @@ static void t_reftable_ref_record_comparison(void)\n {\n \tstruct reftable_record in[3] = {\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t\t\t.u.ref.refname = (char *) \"refs/heads/master\",\n \t\t\t.u.ref.value_type = REFTABLE_REF_VAL1,\n \t\t},\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t\t\t.u.ref.refname = (char *) \"refs/heads/master\",\n \t\t\t.u.ref.value_type = REFTABLE_REF_DELETION,\n \t\t},\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t\t\t.u.ref.refname = (char *) \"HEAD\",\n \t\t\t.u.ref.value_type = REFTABLE_REF_SYMREF,\n \t\t\t.u.ref.value.symref = (char *) \"refs/heads/master\",\n@@ -141,10 +141,10 @@ static void t_reftable_ref_record_roundtrip(void)\n \n \tfor (int i = REFTABLE_REF_DELETION; i < REFTABLE_NR_REF_VALUETYPES; i++) {\n \t\tstruct reftable_record in = {\n-\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n \t\t\t.u.ref.value_type = i,\n \t\t};\n-\t\tstruct reftable_record out = { .type = BLOCK_TYPE_REF };\n+\t\tstruct reftable_record out = { .type = REFTABLE_BLOCK_TYPE_REF };\n \t\tstruct reftable_buf key = REFTABLE_BUF_INIT;\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n@@ -198,17 +198,17 @@ static void t_reftable_log_record_comparison(void)\n {\n \tstruct reftable_record in[3] = {\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_LOG,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t\t\t.u.log.refname = (char *) \"refs/heads/master\",\n \t\t\t.u.log.update_index = 42,\n \t\t},\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_LOG,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t\t\t.u.log.refname = (char *) \"refs/heads/master\",\n \t\t\t.u.log.update_index = 22,\n \t\t},\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_LOG,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t\t\t.u.log.refname = (char *) \"refs/heads/main\",\n \t\t\t.u.log.update_index = 22,\n \t\t},\n@@ -297,7 +297,7 @@ static void t_reftable_log_record_roundtrip(void)\n \tcheck(!reftable_log_record_is_deletion(&in[2]));\n \n \tfor (size_t i = 0; i < ARRAY_SIZE(in); i++) {\n-\t\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG };\n+\t\tstruct reftable_record rec = { .type = REFTABLE_BLOCK_TYPE_LOG };\n \t\tstruct reftable_buf key = REFTABLE_BUF_INIT;\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n@@ -306,7 +306,7 @@ static void t_reftable_log_record_roundtrip(void)\n \t\t};\n \t\t/* populate out, to check for leaks. */\n \t\tstruct reftable_record out = {\n-\t\t\t.type = BLOCK_TYPE_LOG,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n \t\t\t.u.log = {\n \t\t\t\t.refname = xstrdup(\"old name\"),\n \t\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n@@ -384,21 +384,21 @@ static void t_reftable_obj_record_comparison(void)\n \tuint64_t offsets[] = { 0, 16, 32, 48, 64, 80, 96, 112};\n \tstruct reftable_record in[3] = {\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n \t\t\t.u.obj.hash_prefix = id_bytes,\n \t\t\t.u.obj.hash_prefix_len = 7,\n \t\t\t.u.obj.offsets = offsets,\n \t\t\t.u.obj.offset_len = 8,\n \t\t},\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n \t\t\t.u.obj.hash_prefix = id_bytes,\n \t\t\t.u.obj.hash_prefix_len = 7,\n \t\t\t.u.obj.offsets = offsets,\n \t\t\t.u.obj.offset_len = 5,\n \t\t},\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n \t\t\t.u.obj.hash_prefix = id_bytes,\n \t\t\t.u.obj.hash_prefix_len = 5,\n \t\t},\n@@ -450,13 +450,13 @@ static void t_reftable_obj_record_roundtrip(void)\n \t\t\t.len = sizeof(buffer),\n \t\t};\n \t\tstruct reftable_record in = {\n-\t\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n \t\t\t.u = {\n \t\t\t\t.obj = recs[i],\n \t\t\t},\n \t\t};\n \t\tstruct reftable_buf key = REFTABLE_BUF_INIT;\n-\t\tstruct reftable_record out = { .type = BLOCK_TYPE_OBJ };\n+\t\tstruct reftable_record out = { .type = REFTABLE_BLOCK_TYPE_OBJ };\n \t\tint n, m;\n \t\tuint8_t extra;\n \n@@ -482,17 +482,17 @@ static void t_reftable_index_record_comparison(void)\n {\n \tstruct reftable_record in[3] = {\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_INDEX,\n \t\t\t.u.idx.offset = 22,\n \t\t\t.u.idx.last_key = REFTABLE_BUF_INIT,\n \t\t},\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_INDEX,\n \t\t\t.u.idx.offset = 32,\n \t\t\t.u.idx.last_key = REFTABLE_BUF_INIT,\n \t\t},\n \t\t{\n-\t\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t\t.type = REFTABLE_BLOCK_TYPE_INDEX,\n \t\t\t.u.idx.offset = 32,\n \t\t\t.u.idx.last_key = REFTABLE_BUF_INIT,\n \t\t},\n@@ -523,7 +523,7 @@ static void t_reftable_index_record_comparison(void)\n static void t_reftable_index_record_roundtrip(void)\n {\n \tstruct reftable_record in = {\n-\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.type = REFTABLE_BLOCK_TYPE_INDEX,\n \t\t.u.idx = {\n \t\t\t.offset = 42,\n \t\t\t.last_key = REFTABLE_BUF_INIT,\n@@ -537,7 +537,7 @@ static void t_reftable_index_record_roundtrip(void)\n \tstruct reftable_buf scratch = REFTABLE_BUF_INIT;\n \tstruct reftable_buf key = REFTABLE_BUF_INIT;\n \tstruct reftable_record out = {\n-\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.type = REFTABLE_BLOCK_TYPE_INDEX,\n \t\t.u.idx = { .last_key = REFTABLE_BUF_INIT },\n \t};\n \tint n, m;\ndiff --git a/t/unit-tests/t-reftable-table.c b/t/unit-tests/t-reftable-table.c\nindex ba39cdf9a76..7e1eb533d0b 100644\n--- a/t/unit-tests/t-reftable-table.c\n+++ b/t/unit-tests/t-reftable-table.c\n@@ -106,33 +106,33 @@ static int t_table_block_iterator(void)\n \t\tuint16_t record_count;\n \t} expected_blocks[] = {\n \t\t{\n-\t\t\t.block_type = BLOCK_TYPE_REF,\n+\t\t\t.block_type = REFTABLE_BLOCK_TYPE_REF,\n \t\t\t.header_off = 24,\n \t\t\t.restart_count = 10,\n \t\t\t.record_count = 158,\n \t\t},\n \t\t{\n-\t\t\t.block_type = BLOCK_TYPE_REF,\n+\t\t\t.block_type = REFTABLE_BLOCK_TYPE_REF,\n \t\t\t.restart_count = 10,\n \t\t\t.record_count = 159,\n \t\t},\n \t\t{\n-\t\t\t.block_type = BLOCK_TYPE_REF,\n+\t\t\t.block_type = REFTABLE_BLOCK_TYPE_REF,\n \t\t\t.restart_count = 10,\n \t\t\t.record_count = 159,\n \t\t},\n \t\t{\n-\t\t\t.block_type = BLOCK_TYPE_REF,\n+\t\t\t.block_type = REFTABLE_BLOCK_TYPE_REF,\n \t\t\t.restart_count = 2,\n \t\t\t.record_count = 24,\n \t\t},\n \t\t{\n-\t\t\t.block_type = BLOCK_TYPE_INDEX,\n+\t\t\t.block_type = REFTABLE_BLOCK_TYPE_INDEX,\n \t\t\t.restart_count = 1,\n \t\t\t.record_count = 4,\n \t\t},\n \t\t{\n-\t\t\t.block_type = BLOCK_TYPE_OBJ,\n+\t\t\t.block_type = REFTABLE_BLOCK_TYPE_OBJ,\n \t\t\t.restart_count = 1,\n \t\t\t.record_count = 1,\n \t\t},\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"515779","messageId":"20250407-pks-reftable-polishing-v2-16-316c4ff10571@pks.im","threadId":"63222","inReplyTo":"20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im","subject":"[PATCH v2 16/16] reftable/table: move printing logic into test helper","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-07T13:16:28Z","receivedAt":"2025-04-07T13:16:40Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The logic to print individual blocks in a table is hosted in the\nreftable library. This is only the case due to historical reasons though\nbecause users of the library had no interfaces to read blocks one by\none. Otherwise, printing individual blocks has no place in the reftable\nlibrary given that the format will not be generic in the first place.\n\nWe have now grown a public interface to iterate through blocks contained\nin a table, and thus we can finally move the logic to print them into\nthe test helper.\n\nMove over the logic and refactor it accordingly. Note that the iterator\nalso trivially allows us to access index sections, which we previously\ndidn't print at all. This omission wasn't intentional though, so start\ndumping those sections as well so that we can assert that indices are\nwritten as expected.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n reftable/reftable-table.h         |  3 --\n reftable/table.c                  | 65 ------------------------------------\n t/helper/test-reftable.c          | 69 ++++++++++++++++++++++++++++++++++++++-\n t/t0613-reftable-write-options.sh |  9 +++++\n 4 files changed, 77 insertions(+), 69 deletions(-)\n\ndiff --git a/reftable/reftable-table.h b/reftable/reftable-table.h\nindex 4d61ce6cf77..5f935d02e3b 100644\n--- a/reftable/reftable-table.h\n+++ b/reftable/reftable-table.h\n@@ -97,9 +97,6 @@ uint64_t reftable_table_max_update_index(struct reftable_table *t);\n /* return the min_update_index for a table */\n uint64_t reftable_table_min_update_index(struct reftable_table *t);\n \n-/* print blocks onto stdout for debugging. */\n-int reftable_table_print_blocks(const char *tablename);\n-\n /*\n  * An iterator that iterates through the blocks contained in a given table.\n  */\ndiff --git a/reftable/table.c b/reftable/table.c\nindex 1af39423224..ee831276158 100644\n--- a/reftable/table.c\n+++ b/reftable/table.c\n@@ -740,71 +740,6 @@ uint64_t reftable_table_min_update_index(struct reftable_table *t)\n \treturn t->min_update_index;\n }\n \n-int reftable_table_print_blocks(const char *tablename)\n-{\n-\tstruct {\n-\t\tconst char *name;\n-\t\tint type;\n-\t} sections[] = {\n-\t\t{\n-\t\t\t.name = \"ref\",\n-\t\t\t.type = REFTABLE_BLOCK_TYPE_REF,\n-\t\t},\n-\t\t{\n-\t\t\t.name = \"obj\",\n-\t\t\t.type = REFTABLE_BLOCK_TYPE_OBJ,\n-\t\t},\n-\t\t{\n-\t\t\t.name = \"log\",\n-\t\t\t.type = REFTABLE_BLOCK_TYPE_LOG,\n-\t\t},\n-\t};\n-\tstruct reftable_block_source src = { 0 };\n-\tstruct reftable_table *table = NULL;\n-\tstruct table_iter ti = { 0 };\n-\tsize_t i;\n-\tint err;\n-\n-\terr = reftable_block_source_from_file(&src, tablename);\n-\tif (err < 0)\n-\t\tgoto done;\n-\n-\terr = reftable_table_new(&table, &src, tablename);\n-\tif (err < 0)\n-\t\tgoto done;\n-\n-\ttable_iter_init(&ti, table);\n-\n-\tprintf(\"header:\\n\");\n-\tprintf(\"  block_size: %d\\n\", table->block_size);\n-\n-\tfor (i = 0; i < sizeof(sections) / sizeof(*sections); i++) {\n-\t\terr = table_iter_seek_start(&ti, sections[i].type, 0);\n-\t\tif (err < 0)\n-\t\t\tgoto done;\n-\t\tif (err > 0)\n-\t\t\tcontinue;\n-\n-\t\tprintf(\"%s:\\n\", sections[i].name);\n-\n-\t\twhile (1) {\n-\t\t\tprintf(\"  - length: %u\\n\", ti.block.restart_off);\n-\t\t\tprintf(\"    restarts: %u\\n\", ti.block.restart_count);\n-\n-\t\t\terr = table_iter_next_block(&ti);\n-\t\t\tif (err < 0)\n-\t\t\t\tgoto done;\n-\t\t\tif (err > 0)\n-\t\t\t\tbreak;\n-\t\t}\n-\t}\n-\n-done:\n-\treftable_table_decref(table);\n-\ttable_iter_close(&ti);\n-\treturn err;\n-}\n-\n int reftable_table_iterator_init(struct reftable_table_iterator *it,\n \t\t\t\t struct reftable_table *t)\n {\ndiff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c\nindex f8f1956f4f3..b16c0722c84 100644\n--- a/t/helper/test-reftable.c\n+++ b/t/helper/test-reftable.c\n@@ -2,6 +2,7 @@\n #include \"hash.h\"\n #include \"hex.h\"\n #include \"reftable/system.h\"\n+#include \"reftable/reftable-constants.h\"\n #include \"reftable/reftable-error.h\"\n #include \"reftable/reftable-merged.h\"\n #include \"reftable/reftable-stack.h\"\n@@ -20,6 +21,72 @@ static void print_help(void)\n \t       \"\\n\");\n }\n \n+static int dump_blocks(const char *tablename)\n+{\n+\tstruct reftable_table_iterator ti = { 0 };\n+\tstruct reftable_block_source src = { 0 };\n+\tstruct reftable_table *table = NULL;\n+\tuint8_t section_type = 0;\n+\tint err;\n+\n+\terr = reftable_block_source_from_file(&src, tablename);\n+\tif (err < 0)\n+\t\tgoto done;\n+\n+\terr = reftable_table_new(&table, &src, tablename);\n+\tif (err < 0)\n+\t\tgoto done;\n+\n+\terr = reftable_table_iterator_init(&ti, table);\n+\tif (err < 0)\n+\t\tgoto done;\n+\n+\tprintf(\"header:\\n\");\n+\tprintf(\"  block_size: %d\\n\", table->block_size);\n+\n+\twhile (1) {\n+\t\tconst struct reftable_block *block;\n+\n+\t\terr = reftable_table_iterator_next(&ti, &block);\n+\t\tif (err < 0)\n+\t\t\tgoto done;\n+\t\tif (err > 0)\n+\t\t\tbreak;\n+\n+\t\tif (block->block_type != section_type) {\n+\t\t\tconst char *section;\n+\t\t\tswitch (block->block_type) {\n+\t\t\tcase REFTABLE_BLOCK_TYPE_LOG:\n+\t\t\t\tsection = \"log\";\n+\t\t\t\tbreak;\n+\t\t\tcase REFTABLE_BLOCK_TYPE_REF:\n+\t\t\t\tsection = \"ref\";\n+\t\t\t\tbreak;\n+\t\t\tcase REFTABLE_BLOCK_TYPE_OBJ:\n+\t\t\t\tsection = \"obj\";\n+\t\t\t\tbreak;\n+\t\t\tcase REFTABLE_BLOCK_TYPE_INDEX:\n+\t\t\t\tsection = \"idx\";\n+\t\t\t\tbreak;\n+\t\t\tdefault:\n+\t\t\t\terr = -1;\n+\t\t\t\tgoto done;\n+\t\t\t}\n+\n+\t\t\tsection_type = block->block_type;\n+\t\t\tprintf(\"%s:\\n\", section);\n+\t\t}\n+\n+\t\tprintf(\"  - length: %u\\n\", block->restart_off);\n+\t\tprintf(\"    restarts: %u\\n\", block->restart_count);\n+\t}\n+\n+done:\n+\treftable_table_iterator_release(&ti);\n+\treftable_table_decref(table);\n+\treturn err;\n+}\n+\n static int dump_table(struct reftable_merged_table *mt)\n {\n \tstruct reftable_iterator it = { NULL };\n@@ -184,7 +251,7 @@ int cmd__dump_reftable(int argc, const char **argv)\n \targ = argv[1];\n \n \tif (opt_dump_blocks) {\n-\t\terr = reftable_table_print_blocks(arg);\n+\t\terr = dump_blocks(arg);\n \t} else if (opt_dump_table) {\n \t\terr = dump_reftable(arg);\n \t} else if (opt_dump_stack) {\ndiff --git a/t/t0613-reftable-write-options.sh b/t/t0613-reftable-write-options.sh\nindex e2708e11d5b..e4c7461ce9e 100755\n--- a/t/t0613-reftable-write-options.sh\n+++ b/t/t0613-reftable-write-options.sh\n@@ -93,6 +93,9 @@ test_expect_success 'many refs results in multiple blocks' '\n \t\t    restarts: 3\n \t\t  - length: 3289\n \t\t    restarts: 3\n+\t\tidx:\n+\t\t  - length: 103\n+\t\t    restarts: 1\n \t\tEOF\n \t\ttest-tool dump-reftable -b .git/reftable/*.ref >actual &&\n \t\ttest_cmp expect actual\n@@ -241,6 +244,9 @@ test_expect_success 'object index gets written by default with ref index' '\n \t\t    restarts: 1\n \t\t  - length: 80\n \t\t    restarts: 1\n+\t\tidx:\n+\t\t  - length: 55\n+\t\t    restarts: 2\n \t\tobj:\n \t\t  - length: 11\n \t\t    restarts: 1\n@@ -277,6 +283,9 @@ test_expect_success 'object index can be disabled' '\n \t\t    restarts: 1\n \t\t  - length: 80\n \t\t    restarts: 1\n+\t\tidx:\n+\t\t  - length: 55\n+\t\t    restarts: 2\n \t\tEOF\n \t\ttest-tool dump-reftable -b .git/reftable/*.ref >actual &&\n \t\ttest_cmp expect actual\n\n-- \n2.49.0.604.gff1f9ca942.dirty\n\n"},{"id":"516125","messageId":"3okrcl7bdpy75hzyzpmrp7fluan4n3zvsroioq64id4a2kt3o4@fc6fetilb6y2","threadId":"63222","inReplyTo":"20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im","subject":"Re: [PATCH v2 00/16] reftable: overhaul the API to expose access to blocks","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-04-14T19:42:40Z","receivedAt":"2025-04-14T19:46:40Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/04/07 03:16PM, Patrick Steinhardt wrote:\n> Hi,\n> \n> this patch series is a bigger overhaul of the reftable API. This\n> overhaul has two main motivations:\n> \n>   - The reftable library is now standalone and can be used by code bases\n>     other than Git, like libgit2. This patch series thus renames a\n>     couple of subsystems to have more intuitive names before we gain any\n>     new users.\n> \n>   - Some of the data of reftables isn't accessible at all via public\n>     interfaces. Most importantly, it is not possible to access\n>     individual blocks in a table. While users shouldn't need that access\n>     most of the time, an upcoming usecase that Git itself has is to\n>     implement consistency checks for the reftable backend. Here we'll\n>     want to read through blocks and their respective records one by one\n>     to ensure that they are sane and then iterate through records\n>     contained in these blocks.\n> \n> The patch series is structured as follows:\n> \n>   - Patch 1 is a trivial global refactoring to fix formatting of the\n>     license headers. They have been annoying me for far too long.\n> \n>   - Patches 2 to 8 consolidate and rename a couple of data structures:\n> \n>       - `reftable_reader` becomes `reftable_table`, as it is used to\n>         access an individual table.\n> \n>       - `reftable_block` becomes `reftable_block_data`, as it is only a\n>         simple container for the underlying bytes.\n> \n>       - `reftable_block_reader` becomes `reftable_block`, as it is used\n>         to access an individual block.\n> \n>     Overall, the data structures are now called after what they provide\n>     access to compared to the rather generic previous names. This is\n>     also in line with other data structures like `reftable_merged_table`\n>     and `reftable_stack`.\n> \n>   - Patches 9 to 13 refactor the block interface so that it can expose a\n>     generic `reftable_iterator`, granting generic access to all of its\n>     contained records.\n> \n>   - Patches 14 to 16 refactor the table interface to expose a new\n>     iterator over its contained blocks.\n> \n>   - Patch 17 refactors `reftable_table_print_blocks` to be implemented\n>     on top of these new iterators. This allows us to move it out of the\n>     library codebase into the test helper.\n> \n> The series is built on Git v2.49.0 with ps/reftable-sans-compat-util at\n> 8f6a2dbe340 (Makefile: skip reftable library for Coccinelle, 2025-02-18)\n> merged into it.\n> \n> Changes in v2:\n>   - Rename `reftable_table_init_table_iterator()` to\n>     `reftable_table_iterator_init()`.\n>   - Rename `struct reftable_block_reader::block` to `block_data`.\n>   - Several improvements to commit messages.\n>   - Drop unneeded calls to `block_iter_seek_start()` after\n>     `block_iter_init()`.\n>   - Fix -Wcomma error.\n>   - Link to v1: https://lore.kernel.org/r/20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im\n> \n\nThanks Patrick! The changes in this version look good to me.\n\n-Justin\n"}]}