{"thread":{"id":"57047","subject":"[PATCH 00/10] Reftable coverity fixes","startedAt":"2021-12-07T17:45:29Z","lastAt":"2022-03-29T15:59:46Z","messageCount":194,"participants":["Han-Wen Nienhuys via GitGitGadget","Junio C Hamano","Jeff King","Han-Wen Nienhuys","Derrick Stolee","Ævar Arnfjörð Bjarmason","René Scharfe","brian m. carlson","Johannes Schindelin","Neeraj Singh"],"isPatch":true,"patchVersion":1,"patchTotal":10},"messages":[{"id":"443301","messageId":"pull.1152.git.git.1638899124.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":null,"subject":"[PATCH 00/10] Reftable coverity fixes","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-07T17:45:14Z","receivedAt":"2021-12-07T17:45:29Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"This series was targeted to 'next'.\n\nThis addresses some complaints from Coverity that Peff reported.\n\nCI on GGG shows a segfault on windows\n(https://github.com/git/git/runs/4446635428?check_suite_focus=true), which I\ncan't make sense of. Any hints would be greatly appreciated.\n\nHan-Wen Nienhuys (10):\n reftable: fix OOB stack write in print functions\n reftable: fix resource leak in error path\n reftable: fix resource leak blocksource.c\n reftable: check reftable_stack_auto_compact() return value\n reftable: ignore remove() return value in stack_test.c\n reftable: fix resource warning\n reftable: fix NULL derefs in error paths\n reftable: order unittests by complexity\n reftable: drop stray printf in readwrite_test\n reftable: make reftable_record a tagged union\n\n reftable/block.c | 29 ++++--\n reftable/block_test.c | 22 ++---\n reftable/blocksource.c | 6 +-\n reftable/generic.c | 35 ++++----\n reftable/iter.c | 4 +-\n reftable/merged.c | 37 ++++----\n reftable/pq.c | 3 +-\n reftable/pq_test.c | 31 +++----\n reftable/reader.c | 108 ++++++++++++-----------\n reftable/readwrite_test.c | 1 -\n reftable/record.c | 180 +++++++++++++++-----------------------\n reftable/record.h | 45 ++++------\n reftable/record_test.c | 162 +++++++++++++++++-----------------\n reftable/stack.c | 15 ++--\n reftable/stack_test.c | 3 +-\n reftable/writer.c | 46 +++++-----\n t/helper/test-reftable.c | 9 +-\n 17 files changed, 360 insertions(+), 376 deletions(-)\n\n\nbase-commit: a8338297339b1ab064e799c15c4fc56c122ef967\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1152%2Fhanwen%2Freftable-coverity-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1152/hanwen/reftable-coverity-v1\nPull-Request: https://github.com/git/git/pull/1152\n-- \ngitgitgadget\n"},{"id":"443302","messageId":"7c033815183cc8ad7a76bc2b9456b17041107c2c.1638899124.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.git.git.1638899124.gitgitgadget@gmail.com","subject":"[PATCH 01/10] reftable: fix OOB stack write in print functions","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-07T17:45:15Z","receivedAt":"2021-12-07T17:45:30Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 6a5dac32dc6..8536bd03aa9 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -254,7 +254,7 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)\n void reftable_ref_record_print(struct reftable_ref_record *ref,\n \t\t\t uint32_t hash_id)\n {\n-\tchar hex[2 * GIT_SHA256_RAWSZ + 1] = { 0 }; /* BUG */\n+\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */\n \tprintf(\"ref{%s(%\" PRIu64 \") \", ref->refname, ref->update_index);\n \tswitch (ref->value_type) {\n \tcase REFTABLE_REF_SYMREF:\n@@ -586,7 +586,7 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {\n void reftable_log_record_print(struct reftable_log_record *log,\n \t\t\t uint32_t hash_id)\n {\n-\tchar hex[GIT_SHA256_RAWSZ + 1] = { 0 };\n+\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 };\n \n \tswitch (log->value_type) {\n \tcase REFTABLE_LOG_DELETION:\n-- \ngitgitgadget\n\n"},{"id":"443303","messageId":"603bd1d4f6e4ceda02a5165eedd3ae5b92b8929e.1638899124.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.git.git.1638899124.gitgitgadget@gmail.com","subject":"[PATCH 02/10] reftable: fix resource leak in error path","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-07T17:45:16Z","receivedAt":"2021-12-07T17:45:31Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis would be triggered by corrupt files, so it doesn't have test coverage. This\nwas discovered by a Coverity scan.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/block.c | 25 ++++++++++++++++++-------\n 1 file changed, 18 insertions(+), 7 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 855e3f5c947..d7347bb3152 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -188,13 +188,16 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tuint32_t full_block_size = table_block_size;\n \tuint8_t typ = block->data[header_off];\n \tuint32_t sz = get_be24(block->data + header_off + 1);\n-\n+\tint err = 0;\n \tuint16_t restart_count = 0;\n \tuint32_t restart_start = 0;\n \tuint8_t *restart_bytes = NULL;\n+\tuint8_t *uncompressed = NULL;\n \n-\tif (!reftable_is_block_type(typ))\n-\t\treturn REFTABLE_FORMAT_ERROR;\n+\tif (!reftable_is_block_type(typ)) {\n+\t\terr = REFTABLE_FORMAT_ERROR;\n+\t\tgoto done;\n+\t}\n \n \tif (typ == BLOCK_TYPE_LOG) {\n \t\tint block_header_skip = 4 + header_off;\n@@ -213,15 +216,19 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\t uncompress2(uncompressed + block_header_skip, &dst_len,\n \t\t\t\tblock->data + block_header_skip, &src_len)) {\n \t\t\treftable_free(uncompressed);\n-\t\t\treturn REFTABLE_ZLIB_ERROR;\n+\t\t\terr = REFTABLE_ZLIB_ERROR;\n+\t\t\tgoto done;\n \t\t}\n \n-\t\tif (dst_len + block_header_skip != sz)\n-\t\t\treturn REFTABLE_FORMAT_ERROR;\n+\t\tif (dst_len + block_header_skip != sz) {\n+\t\t\terr = REFTABLE_FORMAT_ERROR;\n+\t\t\tgoto done;\n+\t\t}\n \n \t\t/* We're done with the input data. */\n \t\treftable_block_done(block);\n \t\tblock->data = uncompressed;\n+\t\tuncompressed = NULL;\n \t\tblock->len = sz;\n \t\tblock->source = malloc_block_source();\n \t\tfull_block_size = src_len + block_header_skip;\n@@ -251,7 +258,11 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tbr->restart_count = restart_count;\n \tbr->restart_bytes = restart_bytes;\n \n-\treturn 0;\n+done:\n+\tif (uncompressed) {\n+\t\treftable_free(uncompressed);\n+\t}\n+\treturn err;\n }\n \n static uint32_t block_reader_restart_offset(struct block_reader *br, int i)\n-- \ngitgitgadget\n\n"},{"id":"443304","messageId":"97279040dded376c110299cc8f2a6a9f7fc4950a.1638899124.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.git.git.1638899124.gitgitgadget@gmail.com","subject":"[PATCH 03/10] reftable: fix resource leak blocksource.c","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-07T17:45:17Z","receivedAt":"2021-12-07T17:45:32Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis would be triggered in the unlikely event of fstat() failing on an opened\nfile.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/blocksource.c | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/reftable/blocksource.c b/reftable/blocksource.c\nindex 0044eecd9aa..2605371c28d 100644\n--- a/reftable/blocksource.c\n+++ b/reftable/blocksource.c\n@@ -134,8 +134,10 @@ int reftable_block_source_from_file(struct reftable_block_source *bs,\n \t}\n \n \terr = fstat(fd, &st);\n-\tif (err < 0)\n-\t\treturn -1;\n+\tif (err < 0) {\n+\t\tclose(fd);\n+\t\treturn REFTABLE_IO_ERROR;\n+\t}\n \n \tp = reftable_calloc(sizeof(struct file_block_source));\n \tp->size = st.st_size;\n-- \ngitgitgadget\n\n"},{"id":"443305","messageId":"9288289ebcb7f7db0bfeef80f3150e1205dc1d62.1638899124.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.git.git.1638899124.gitgitgadget@gmail.com","subject":"[PATCH 04/10] reftable: check reftable_stack_auto_compact() return value","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-07T17:45:18Z","receivedAt":"2021-12-07T17:45:33Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nFixes a problem detected by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack_test.c | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/reftable/stack_test.c b/reftable/stack_test.c\nindex eb0b7228b0c..d628420e63a 100644\n--- a/reftable/stack_test.c\n+++ b/reftable/stack_test.c\n@@ -814,6 +814,7 @@ static void test_reftable_stack_auto_compaction(void)\n \t\tEXPECT_ERR(err);\n \n \t\terr = reftable_stack_auto_compact(st);\n+\t\tEXPECT_ERR(err);\n \t\tEXPECT(i < 3 || st->merged->stack_len < 2 * fastlog2(i));\n \t}\n \n-- \ngitgitgadget\n\n"},{"id":"443306","messageId":"9269977a1c3c2654afbc6ae18e9bb442fbc7968a.1638899124.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.git.git.1638899124.gitgitgadget@gmail.com","subject":"[PATCH 05/10] reftable: ignore remove() return value in stack_test.c","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-07T17:45:19Z","receivedAt":"2021-12-07T17:45:35Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nIf the cleanup fails, there is nothing we can do.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack_test.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/reftable/stack_test.c b/reftable/stack_test.c\nindex d628420e63a..4b7292945c3 100644\n--- a/reftable/stack_test.c\n+++ b/reftable/stack_test.c\n@@ -89,7 +89,7 @@ static void test_read_file(void)\n \t\tEXPECT(0 == strcmp(want[i], names[i]));\n \t}\n \tfree_names(names);\n-\tremove(fn);\n+\t(void) remove(fn);\n }\n \n static void test_parse_names(void)\n-- \ngitgitgadget\n\n"},{"id":"443307","messageId":"eab71ab008e690ddb06d8ec5c27257b90ef26d2a.1638899124.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.git.git.1638899124.gitgitgadget@gmail.com","subject":"[PATCH 06/10] reftable: fix resource warning","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-07T17:45:20Z","receivedAt":"2021-12-07T17:45:35Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis would trigger in the unlikely event that we are compacting, and the next\navailable file handle is 0.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack.c | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/reftable/stack.c b/reftable/stack.c\nindex df5021ebf08..10dfd370e8e 100644\n--- a/reftable/stack.c\n+++ b/reftable/stack.c\n@@ -877,7 +877,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \tstruct strbuf new_table_path = STRBUF_INIT;\n \tint err = 0;\n \tint have_lock = 0;\n-\tint lock_file_fd = 0;\n+\tint lock_file_fd = -1;\n \tint compact_count = last - first + 1;\n \tchar **listp = NULL;\n \tchar **delete_on_success =\n@@ -911,7 +911,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \t}\n \t/* Don't want to write to the lock for now. */\n \tclose(lock_file_fd);\n-\tlock_file_fd = 0;\n+\tlock_file_fd = -1;\n \n \thave_lock = 1;\n \terr = stack_uptodate(st);\n@@ -932,7 +932,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \n \t\tsublock_file_fd = open(subtab_lock.buf,\n \t\t\t\t O_EXCL | O_CREAT | O_WRONLY, 0644);\n-\t\tif (sublock_file_fd > 0) {\n+\t\tif (sublock_file_fd >= 0) {\n \t\t\tclose(sublock_file_fd);\n \t\t} else if (sublock_file_fd < 0) {\n \t\t\tif (errno == EEXIST) {\n@@ -1013,7 +1013,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \t\tgoto done;\n \t}\n \terr = close(lock_file_fd);\n-\tlock_file_fd = 0;\n+\tlock_file_fd = -1;\n \tif (err < 0) {\n \t\terr = REFTABLE_IO_ERROR;\n \t\tunlink(new_table_path.buf);\n@@ -1050,9 +1050,9 @@ done:\n \t\tlistp++;\n \t}\n \tfree_names(subtable_locks);\n-\tif (lock_file_fd > 0) {\n+\tif (lock_file_fd >= 0) {\n \t\tclose(lock_file_fd);\n-\t\tlock_file_fd = 0;\n+\t\tlock_file_fd = -1;\n \t}\n \tif (have_lock) {\n \t\tunlink(lock_file_name.buf);\n-- \ngitgitgadget\n\n"},{"id":"443308","messageId":"47fef87eb4a614fb53d703152079cdd64b623814.1638899124.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.git.git.1638899124.gitgitgadget@gmail.com","subject":"[PATCH 07/10] reftable: fix NULL derefs in error paths","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-07T17:45:21Z","receivedAt":"2021-12-07T17:45:36Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSpotted by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/reader.c | 3 ++-\n reftable/stack.c | 3 ++-\n 2 files changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 006709a645a..8d308d858f8 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -796,6 +796,7 @@ int reftable_reader_print_file(const char *tablename)\n \treftable_table_from_reader(&tab, r);\n \terr = reftable_table_print(&tab);\n done:\n-\treftable_reader_free(r);\n+\tif (r)\n+\t\treftable_reader_free(r);\n \treturn err;\n }\ndiff --git a/reftable/stack.c b/reftable/stack.c\nindex 10dfd370e8e..eb03b6c74f6 100644\n--- a/reftable/stack.c\n+++ b/reftable/stack.c\n@@ -707,7 +707,8 @@ done:\n \tstrbuf_release(&temp_tab_file_name);\n \tstrbuf_release(&tab_file_name);\n \tstrbuf_release(&next_name);\n-\treftable_writer_free(wr);\n+\tif (wr)\n+\t\treftable_writer_free(wr);\n \treturn err;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"443309","messageId":"458adf0fa9b90dae831da7aa8cdb67fa9a064ab5.1638899124.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.git.git.1638899124.gitgitgadget@gmail.com","subject":"[PATCH 08/10] reftable: order unittests by complexity","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-07T17:45:22Z","receivedAt":"2021-12-07T17:45:37Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis is a more pratical ordering when working on refactorings of the reftable\ncode.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n t/helper/test-reftable.c | 9 +++++----\n 1 file changed, 5 insertions(+), 4 deletions(-)\n\ndiff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c\nindex 26b03d7b789..f08d66df949 100644\n--- a/t/helper/test-reftable.c\n+++ b/t/helper/test-reftable.c\n@@ -3,15 +3,16 @@\n \n int cmd__reftable(int argc, const char **argv)\n {\n+\t// test from simple to complex.\n \tbasics_test_main(argc, argv);\n+\trecord_test_main(argc, argv);\n \tblock_test_main(argc, argv);\n-\tmerged_test_main(argc, argv);\n+\ttree_test_main(argc, argv);\n \tpq_test_main(argc, argv);\n-\trecord_test_main(argc, argv);\n-\trefname_test_main(argc, argv);\n \treadwrite_test_main(argc, argv);\n+\tmerged_test_main(argc, argv);\n \tstack_test_main(argc, argv);\n-\ttree_test_main(argc, argv);\n+\trefname_test_main(argc, argv);\n \treturn 0;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"443310","messageId":"cd5d8d3607da67011c00b1c96a34a9ed30f541a8.1638899124.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.git.git.1638899124.gitgitgadget@gmail.com","subject":"[PATCH 09/10] reftable: drop stray printf in readwrite_test","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-07T17:45:23Z","receivedAt":"2021-12-07T17:45:41Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/readwrite_test.c | 1 -\n 1 file changed, 1 deletion(-)\n\ndiff --git a/reftable/readwrite_test.c b/reftable/readwrite_test.c\nindex 5f6bcc2f775..7c40b9b77f3 100644\n--- a/reftable/readwrite_test.c\n+++ b/reftable/readwrite_test.c\n@@ -597,7 +597,6 @@ static void test_write_key_order(void)\n \terr = reftable_writer_add_ref(w, &refs[0]);\n \tEXPECT_ERR(err);\n \terr = reftable_writer_add_ref(w, &refs[1]);\n-\tprintf(\"%d\\n\", err);\n \tEXPECT(err == REFTABLE_API_ERROR);\n \treftable_writer_close(w);\n \treftable_writer_free(w);\n-- \ngitgitgadget\n\n"},{"id":"443311","messageId":"8deccc3a1dff7e4f7d613fa63d2781fd1f11f841.1638899124.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.git.git.1638899124.gitgitgadget@gmail.com","subject":"[PATCH 10/10] reftable: make reftable_record a tagged union","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-07T17:45:24Z","receivedAt":"2021-12-07T17:45:42Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis reduces the amount of glue code, because we don't need a void pointer or\nvtable within the structure.\n\nThe only snag is that reftable_index_record contain a strbuf, so it cannot be\nzero-initialized. To address this, introduce reftable_record_for() to create a\nfresh instance, given a record type.\n\nThanks to Peff for the suggestion.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/block.c | 4 +-\n reftable/block_test.c | 22 +++---\n reftable/generic.c | 35 ++++----\n reftable/iter.c | 4 +-\n reftable/merged.c | 37 ++++-----\n reftable/pq.c | 3 +-\n reftable/pq_test.c | 31 ++++----\n reftable/reader.c | 105 ++++++++++++------------\n reftable/record.c | 176 ++++++++++++++++-------------------------\n reftable/record.h | 45 +++++------\n reftable/record_test.c | 162 +++++++++++++++++++------------------\n reftable/writer.c | 46 ++++++-----\n 12 files changed, 319 insertions(+), 351 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex d7347bb3152..ad713caa57b 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -382,7 +382,7 @@ int block_reader_seek(struct block_reader *br, struct block_iter *it,\n \t\t.key = *want,\n \t\t.r = br,\n \t};\n-\tstruct reftable_record rec = reftable_new_record(block_reader_type(br));\n+\tstruct reftable_record rec = reftable_record_for(block_reader_type(br));\n \tstruct strbuf key = STRBUF_INIT;\n \tint err = 0;\n \tstruct block_iter next = {\n@@ -424,7 +424,7 @@ int block_reader_seek(struct block_reader *br, struct block_iter *it,\n done:\n \tstrbuf_release(&key);\n \tstrbuf_release(&next.last_key);\n-\treftable_record_destroy(&rec);\n+\treftable_record_release(&rec);\n \n \treturn err;\n }\ndiff --git a/reftable/block_test.c b/reftable/block_test.c\nindex 4b3ea262dcb..670329f2224 100644\n--- a/reftable/block_test.c\n+++ b/reftable/block_test.c\n@@ -26,8 +26,9 @@ static void test_block_read_write(void)\n \tstruct block_writer bw = {\n \t\t.last_key = STRBUF_INIT,\n \t};\n-\tstruct reftable_ref_record ref = { NULL };\n-\tstruct reftable_record rec = { NULL };\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t};\n \tint i = 0;\n \tint n;\n \tstruct block_reader br = { 0 };\n@@ -40,7 +41,6 @@ static void test_block_read_write(void)\n \tblock.source = malloc_block_source();\n \tblock_writer_init(&bw, BLOCK_TYPE_REF, block.data, block_size,\n \t\t\t header_off, hash_size(GIT_SHA1_FORMAT_ID));\n-\treftable_record_from_ref(&rec, &ref);\n \n \tfor (i = 0; i < N; i++) {\n \t\tchar name[100];\n@@ -48,14 +48,14 @@ static void test_block_read_write(void)\n \t\tsnprintf(name, sizeof(name), \"branch%02d\", i);\n \t\tmemset(hash, i, sizeof(hash));\n \n-\t\tref.refname = name;\n-\t\tref.value_type = REFTABLE_REF_VAL1;\n-\t\tref.value.val1 = hash;\n+\t\trec.ref.refname = name;\n+\t\trec.ref.value_type = REFTABLE_REF_VAL1;\n+\t\trec.ref.value.val1 = hash;\n \n \t\tnames[i] = xstrdup(name);\n \t\tn = block_writer_add(&bw, &rec);\n-\t\tref.refname = NULL;\n-\t\tref.value_type = REFTABLE_REF_DELETION;\n+\t\trec.ref.refname = NULL;\n+\t\trec.ref.value_type = REFTABLE_REF_DELETION;\n \t\tEXPECT(n == 0);\n \t}\n \n@@ -74,7 +74,7 @@ static void test_block_read_write(void)\n \t\tif (r > 0) {\n \t\t\tbreak;\n \t\t}\n-\t\tEXPECT_STREQ(names[j], ref.refname);\n+\t\tEXPECT_STREQ(names[j], rec.ref.refname);\n \t\tj++;\n \t}\n \n@@ -92,7 +92,7 @@ static void test_block_read_write(void)\n \t\tn = block_iter_next(&it, &rec);\n \t\tEXPECT(n == 0);\n \n-\t\tEXPECT_STREQ(names[i], ref.refname);\n+\t\tEXPECT_STREQ(names[i], rec.ref.refname);\n \n \t\twant.len--;\n \t\tn = block_reader_seek(&br, &it, &want);\n@@ -100,7 +100,7 @@ static void test_block_read_write(void)\n \n \t\tn = block_iter_next(&it, &rec);\n \t\tEXPECT(n == 0);\n-\t\tEXPECT_STREQ(names[10 * (i / 10)], ref.refname);\n+\t\tEXPECT_STREQ(names[10 * (i / 10)], rec.ref.refname);\n \n \t\tblock_iter_close(&it);\n \t}\ndiff --git a/reftable/generic.c b/reftable/generic.c\nindex 7a8a738d860..7f00aa9cdbe 100644\n--- a/reftable/generic.c\n+++ b/reftable/generic.c\n@@ -7,6 +7,7 @@ https://developers.google.com/open-source/licenses/bsd\n */\n \n #include \"basics.h\"\n+#include \"constants.h\"\n #include \"record.h\"\n #include \"generic.h\"\n #include \"reftable-iterator.h\"\n@@ -15,23 +16,21 @@ https://developers.google.com/open-source/licenses/bsd\n int reftable_table_seek_ref(struct reftable_table *tab,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_REF,\n+\t\t\t\t .ref = {\n+\t\t\t\t\t .refname = (char *)name,\n+\t\t\t\t } };\n \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n }\n \n int reftable_table_seek_log(struct reftable_table *tab,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = ~((uint64_t)0),\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n+\t\t\t\t .log = {\n+\t\t\t\t\t .refname = (char *)name,\n+\t\t\t\t\t .update_index = ~((uint64_t)0),\n+\t\t\t\t } };\n \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n }\n \n@@ -129,17 +128,19 @@ void reftable_iterator_destroy(struct reftable_iterator *it)\n int reftable_iterator_next_ref(struct reftable_iterator *it,\n \t\t\t struct reftable_ref_record *ref)\n {\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, ref);\n-\treturn iterator_next(it, &rec);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_REF };\n+\tint err = iterator_next(it, &rec);\n+\t*ref = rec.ref;\n+\treturn err;\n }\n \n int reftable_iterator_next_log(struct reftable_iterator *it,\n \t\t\t struct reftable_log_record *log)\n {\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, log);\n-\treturn iterator_next(it, &rec);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG };\n+\tint err = iterator_next(it, &rec);\n+\t*log = rec.log;\n+\treturn err;\n }\n \n int iterator_next(struct reftable_iterator *it, struct reftable_record *rec)\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex 93d04f735b8..fedf5d8ab0b 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -32,7 +32,7 @@ static int filtering_ref_iterator_next(void *iter_arg,\n \t\t\t\t struct reftable_record *rec)\n {\n \tstruct filtering_ref_iterator *fri = iter_arg;\n-\tstruct reftable_ref_record *ref = rec->data;\n+\tstruct reftable_ref_record *ref = &rec->ref;\n \tint err = 0;\n \twhile (1) {\n \t\terr = reftable_iterator_next_ref(&fri->it, ref);\n@@ -127,7 +127,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n static int indexed_table_ref_iter_next(void *p, struct reftable_record *rec)\n {\n \tstruct indexed_table_ref_iter *it = p;\n-\tstruct reftable_ref_record *ref = rec->data;\n+\tstruct reftable_ref_record *ref = &rec->ref;\n \n \twhile (1) {\n \t\tint err = block_iter_next(&it->cur, rec);\ndiff --git a/reftable/merged.c b/reftable/merged.c\nindex e5b53da6db3..e69955d7c2d 100644\n--- a/reftable/merged.c\n+++ b/reftable/merged.c\n@@ -22,7 +22,7 @@ static int merged_iter_init(struct merged_iter *mi)\n {\n \tint i = 0;\n \tfor (i = 0; i < mi->stack_len; i++) {\n-\t\tstruct reftable_record rec = reftable_new_record(mi->typ);\n+\t\tstruct reftable_record rec = reftable_record_for(mi->typ);\n \t\tint err = iterator_next(&mi->stack[i], &rec);\n \t\tif (err < 0) {\n \t\t\treturn err;\n@@ -30,7 +30,7 @@ static int merged_iter_init(struct merged_iter *mi)\n \n \t\tif (err > 0) {\n \t\t\treftable_iterator_destroy(&mi->stack[i]);\n-\t\t\treftable_record_destroy(&rec);\n+\t\t\treftable_record_release(&rec);\n \t\t} else {\n \t\t\tstruct pq_entry e = {\n \t\t\t\t.rec = rec,\n@@ -57,18 +57,17 @@ static void merged_iter_close(void *p)\n static int merged_iter_advance_nonnull_subiter(struct merged_iter *mi,\n \t\t\t\t\t size_t idx)\n {\n-\tstruct reftable_record rec = reftable_new_record(mi->typ);\n \tstruct pq_entry e = {\n-\t\t.rec = rec,\n+\t\t.rec = reftable_record_for(mi->typ),\n \t\t.index = idx,\n \t};\n-\tint err = iterator_next(&mi->stack[idx], &rec);\n+\tint err = iterator_next(&mi->stack[idx], &e.rec);\n \tif (err < 0)\n \t\treturn err;\n \n \tif (err > 0) {\n \t\treftable_iterator_destroy(&mi->stack[idx]);\n-\t\treftable_record_destroy(&rec);\n+\t\treftable_record_release(&e.rec);\n \t\treturn 0;\n \t}\n \n@@ -126,11 +125,11 @@ static int merged_iter_next_entry(struct merged_iter *mi,\n \t\tif (err < 0) {\n \t\t\treturn err;\n \t\t}\n-\t\treftable_record_destroy(&top.rec);\n+\t\treftable_record_release(&top.rec);\n \t}\n \n \treftable_record_copy_from(rec, &entry.rec, hash_size(mi->hash_id));\n-\treftable_record_destroy(&entry.rec);\n+\treftable_record_release(&entry.rec);\n \tstrbuf_release(&entry_key);\n \treturn 0;\n }\n@@ -246,7 +245,7 @@ static int merged_table_seek_record(struct reftable_merged_table *mt,\n \t\tsizeof(struct reftable_iterator) * mt->stack_len);\n \tstruct merged_iter merged = {\n \t\t.stack = iters,\n-\t\t.typ = reftable_record_type(rec),\n+\t\t.typ = rec->type,\n \t\t.hash_id = mt->hash_id,\n \t\t.suppress_deletions = mt->suppress_deletions,\n \t};\n@@ -290,11 +289,12 @@ int reftable_merged_table_seek_ref(struct reftable_merged_table *mt,\n \t\t\t\t struct reftable_iterator *it,\n \t\t\t\t const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.ref = {\n+\t\t\t.refname = (char *)name,\n+\t\t},\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n \treturn merged_table_seek_record(mt, it, &rec);\n }\n \n@@ -302,12 +302,13 @@ int reftable_merged_table_seek_log_at(struct reftable_merged_table *mt,\n \t\t\t\t struct reftable_iterator *it,\n \t\t\t\t const char *name, uint64_t update_index)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = update_index,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.log = {\n+\t\t\t.refname = (char *)name,\n+\t\t\t.update_index = update_index,\n+\t\t}\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n \treturn merged_table_seek_record(mt, it, &rec);\n }\n \ndiff --git a/reftable/pq.c b/reftable/pq.c\nindex efc474017a2..96ca6dd37b3 100644\n--- a/reftable/pq.c\n+++ b/reftable/pq.c\n@@ -74,6 +74,7 @@ struct pq_entry merged_iter_pqueue_remove(struct merged_iter_pqueue *pq)\n void merged_iter_pqueue_add(struct merged_iter_pqueue *pq, struct pq_entry e)\n {\n \tint i = 0;\n+\n \tif (pq->len == pq->cap) {\n \t\tpq->cap = 2 * pq->cap + 1;\n \t\tpq->heap = reftable_realloc(pq->heap,\n@@ -98,7 +99,7 @@ void merged_iter_pqueue_release(struct merged_iter_pqueue *pq)\n {\n \tint i = 0;\n \tfor (i = 0; i < pq->len; i++) {\n-\t\treftable_record_destroy(&pq->heap[i].rec);\n+\t\treftable_record_release(&pq->heap[i].rec);\n \t}\n \tFREE_AND_NULL(pq->heap);\n \tpq->len = pq->cap = 0;\ndiff --git a/reftable/pq_test.c b/reftable/pq_test.c\nindex c9bb05e37b7..89d069d674d 100644\n--- a/reftable/pq_test.c\n+++ b/reftable/pq_test.c\n@@ -31,7 +31,7 @@ static void test_pq(void)\n \tint N = ARRAY_SIZE(names) - 1;\n \n \tstruct merged_iter_pqueue pq = { NULL };\n-\tconst char *last = NULL;\n+\tchar *last = NULL;\n \n \tint i = 0;\n \tfor (i = 0; i < N; i++) {\n@@ -42,12 +42,14 @@ static void test_pq(void)\n \n \ti = 1;\n \tdo {\n-\t\tstruct reftable_record rec =\n-\t\t\treftable_new_record(BLOCK_TYPE_REF);\n-\t\tstruct pq_entry e = { 0 };\n-\n-\t\treftable_record_as_ref(&rec)->refname = names[i];\n-\t\te.rec = rec;\n+\t\tstruct pq_entry e = {\n+\t\t\t.rec = {\n+\t\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t\t.ref = {\n+\t\t\t\t\t.refname = names[i],\n+\t\t\t\t}\n+\t\t\t}\n+\t\t};\n \t\tmerged_iter_pqueue_add(&pq, e);\n \t\tmerged_iter_pqueue_check(pq);\n \t\ti = (i * 7) % N;\n@@ -55,19 +57,18 @@ static void test_pq(void)\n \n \twhile (!merged_iter_pqueue_is_empty(pq)) {\n \t\tstruct pq_entry e = merged_iter_pqueue_remove(&pq);\n-\t\tstruct reftable_ref_record *ref =\n-\t\t\treftable_record_as_ref(&e.rec);\n-\n+\t\tstruct reftable_record *rec = &e.rec;\n \t\tmerged_iter_pqueue_check(pq);\n \n+\t\tEXPECT(rec->type == BLOCK_TYPE_REF);\n \t\tif (last) {\n-\t\t\tEXPECT(strcmp(last, ref->refname) < 0);\n+\t\t\tEXPECT(strcmp(last, rec->ref.refname) < 0);\n \t\t}\n-\t\tlast = ref->refname;\n-\t\tref->refname = NULL;\n-\t\treftable_free(ref);\n+\t\t// this is names[i], so don't dealloc.\n+\t\tlast = rec->ref.refname;\n+\t\trec->ref.refname = NULL;\n+\t\treftable_record_release(rec);\n \t}\n-\n \tfor (i = 0; i < N; i++) {\n \t\treftable_free(names[i]);\n \t}\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 8d308d858f8..0159f131122 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -238,9 +238,8 @@ static int table_iter_next_in_block(struct table_iter *ti,\n \t\t\t\t struct reftable_record *rec)\n {\n \tint res = block_iter_next(&ti->bi, rec);\n-\tif (res == 0 && reftable_record_type(rec) == BLOCK_TYPE_REF) {\n-\t\t((struct reftable_ref_record *)rec->data)->update_index +=\n-\t\t\tti->r->min_update_index;\n+\tif (res == 0 && rec->type == BLOCK_TYPE_REF) {\n+\t\trec->ref.update_index += ti->r->min_update_index;\n \t}\n \n \treturn res;\n@@ -345,7 +344,7 @@ static int table_iter_next_block(struct table_iter *dest,\n \n static int table_iter_next(struct table_iter *ti, struct reftable_record *rec)\n {\n-\tif (reftable_record_type(rec) != ti->typ)\n+\tif (rec->type != ti->typ)\n \t\treturn REFTABLE_API_ERROR;\n \n \twhile (1) {\n@@ -437,8 +436,7 @@ static int reader_start(struct reftable_reader *r, struct table_iter *ti,\n static int reader_seek_linear(struct reftable_reader *r, struct table_iter *ti,\n \t\t\t struct reftable_record *want)\n {\n-\tstruct reftable_record rec =\n-\t\treftable_new_record(reftable_record_type(want));\n+\tstruct reftable_record rec = reftable_record_for(want->type);\n \tstruct strbuf want_key = STRBUF_INIT;\n \tstruct strbuf got_key = STRBUF_INIT;\n \tstruct table_iter next = TABLE_ITER_INIT;\n@@ -475,7 +473,7 @@ static int reader_seek_linear(struct reftable_reader *r, struct table_iter *ti,\n \n done:\n \tblock_iter_close(&next.bi);\n-\treftable_record_destroy(&rec);\n+\treftable_record_release(&rec);\n \tstrbuf_release(&want_key);\n \tstrbuf_release(&got_key);\n \treturn err;\n@@ -485,38 +483,40 @@ static int reader_seek_indexed(struct reftable_reader *r,\n \t\t\t struct reftable_iterator *it,\n \t\t\t struct reftable_record *rec)\n {\n-\tstruct reftable_index_record want_index = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record want_index_rec = { NULL };\n-\tstruct reftable_index_record index_result = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record index_result_rec = { NULL };\n+\tstruct reftable_record want_index = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.idx = { .last_key = STRBUF_INIT }\n+\t};\n+\tstruct reftable_record index_result = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.idx = { .last_key = STRBUF_INIT },\n+\t};\n \tstruct table_iter index_iter = TABLE_ITER_INIT;\n \tstruct table_iter next = TABLE_ITER_INIT;\n \tint err = 0;\n \n-\treftable_record_key(rec, &want_index.last_key);\n-\treftable_record_from_index(&want_index_rec, &want_index);\n-\treftable_record_from_index(&index_result_rec, &index_result);\n-\n-\terr = reader_start(r, &index_iter, reftable_record_type(rec), 1);\n+\treftable_record_key(rec, &want_index.idx.last_key);\n+\terr = reader_start(r, &index_iter, rec->type, 1);\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = reader_seek_linear(r, &index_iter, &want_index_rec);\n+\terr = reader_seek_linear(r, &index_iter, &want_index);\n \twhile (1) {\n-\t\terr = table_iter_next(&index_iter, &index_result_rec);\n+\t\terr = table_iter_next(&index_iter, &index_result);\n \t\ttable_iter_block_done(&index_iter);\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = reader_table_iter_at(r, &next, index_result.offset, 0);\n+\t\terr = reader_table_iter_at(r, &next, index_result.idx.offset,\n+\t\t\t\t\t 0);\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = block_iter_seek(&next.bi, &want_index.last_key);\n+\t\terr = block_iter_seek(&next.bi, &want_index.idx.last_key);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n-\t\tif (next.typ == reftable_record_type(rec)) {\n+\t\tif (next.typ == rec->type) {\n \t\t\terr = 0;\n \t\t\tbreak;\n \t\t}\n@@ -540,8 +540,8 @@ static int reader_seek_indexed(struct reftable_reader *r,\n done:\n \tblock_iter_close(&next.bi);\n \ttable_iter_close(&index_iter);\n-\treftable_record_release(&want_index_rec);\n-\treftable_record_release(&index_result_rec);\n+\treftable_record_release(&want_index);\n+\treftable_record_release(&index_result);\n \treturn err;\n }\n \n@@ -549,15 +549,14 @@ static int reader_seek_internal(struct reftable_reader *r,\n \t\t\t\tstruct reftable_iterator *it,\n \t\t\t\tstruct reftable_record *rec)\n {\n-\tstruct reftable_reader_offsets *offs =\n-\t\treader_offsets_for(r, reftable_record_type(rec));\n+\tstruct reftable_reader_offsets *offs = reader_offsets_for(r, rec->type);\n \tuint64_t idx = offs->index_offset;\n \tstruct table_iter ti = TABLE_ITER_INIT;\n \tint err = 0;\n \tif (idx > 0)\n \t\treturn reader_seek_indexed(r, it, rec);\n \n-\terr = reader_start(r, &ti, reftable_record_type(rec), 0);\n+\terr = reader_start(r, &ti, rec->type, 0);\n \tif (err < 0)\n \t\treturn err;\n \terr = reader_seek_linear(r, &ti, rec);\n@@ -576,7 +575,7 @@ static int reader_seek_internal(struct reftable_reader *r,\n static int reader_seek(struct reftable_reader *r, struct reftable_iterator *it,\n \t\t struct reftable_record *rec)\n {\n-\tuint8_t typ = reftable_record_type(rec);\n+\tuint8_t typ = rec->type;\n \n \tstruct reftable_reader_offsets *offs = reader_offsets_for(r, typ);\n \tif (!offs->is_present) {\n@@ -590,11 +589,12 @@ static int reader_seek(struct reftable_reader *r, struct reftable_iterator *it,\n int reftable_reader_seek_ref(struct reftable_reader *r,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.ref = {\n+\t\t\t.refname = (char *)name,\n+\t\t},\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n \treturn reader_seek(r, it, &rec);\n }\n \n@@ -602,12 +602,13 @@ int reftable_reader_seek_log_at(struct reftable_reader *r,\n \t\t\t\tstruct reftable_iterator *it, const char *name,\n \t\t\t\tuint64_t update_index)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = update_index,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.log = {\n+\t\t\t.refname = (char *)name,\n+\t\t\t.update_index = update_index,\n+\t\t}\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n \treturn reader_seek(r, it, &rec);\n }\n \n@@ -649,31 +650,33 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \t\t\t\t\t struct reftable_iterator *it,\n \t\t\t\t\t uint8_t *oid)\n {\n-\tstruct reftable_obj_record want = {\n-\t\t.hash_prefix = oid,\n-\t\t.hash_prefix_len = r->object_id_len,\n+\tstruct reftable_record want = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.obj = {\n+\t\t\t.hash_prefix = oid,\n+\t\t\t.hash_prefix_len = r->object_id_len,\n+\t\t},\n \t};\n-\tstruct reftable_record want_rec = { NULL };\n \tstruct reftable_iterator oit = { NULL };\n-\tstruct reftable_obj_record got = { NULL };\n-\tstruct reftable_record got_rec = { NULL };\n+\tstruct reftable_record got = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.obj = { 0 },\n+\t};\n \tint err = 0;\n \tstruct indexed_table_ref_iter *itr = NULL;\n \n \t/* Look through the reverse index. */\n-\treftable_record_from_obj(&want_rec, &want);\n-\terr = reader_seek(r, &oit, &want_rec);\n+\terr = reader_seek(r, &oit, &want);\n \tif (err != 0)\n \t\tgoto done;\n \n \t/* read out the reftable_obj_record */\n-\treftable_record_from_obj(&got_rec, &got);\n-\terr = iterator_next(&oit, &got_rec);\n+\terr = iterator_next(&oit, &got);\n \tif (err < 0)\n \t\tgoto done;\n \n-\tif (err > 0 ||\n-\t memcmp(want.hash_prefix, got.hash_prefix, r->object_id_len)) {\n+\tif (err > 0 || memcmp(want.obj.hash_prefix, got.obj.hash_prefix,\n+\t\t\t r->object_id_len)) {\n \t\t/* didn't find it; return empty iterator */\n \t\titerator_set_empty(it);\n \t\terr = 0;\n@@ -681,15 +684,15 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \t}\n \n \terr = new_indexed_table_ref_iter(&itr, r, oid, hash_size(r->hash_id),\n-\t\t\t\t\t got.offsets, got.offset_len);\n+\t\t\t\t\t got.obj.offsets, got.obj.offset_len);\n \tif (err < 0)\n \t\tgoto done;\n-\tgot.offsets = NULL;\n+\tgot.obj.offsets = NULL;\n \titerator_from_indexed_table_ref_iter(it, itr);\n \n done:\n \treftable_iterator_destroy(&oit);\n-\treftable_record_release(&got_rec);\n+\treftable_record_release(&got);\n \treturn err;\n }\n \ndiff --git a/reftable/record.c b/reftable/record.c\nindex 8536bd03aa9..b665feb709f 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -15,6 +15,10 @@ https://developers.google.com/open-source/licenses/bsd\n #include \"reftable-error.h\"\n #include \"basics.h\"\n \n+static struct reftable_record_vtable *\n+reftable_record_vtable(struct reftable_record *rec);\n+static void *reftable_record_data(struct reftable_record *rec);\n+\n int get_var_int(uint64_t *dest, struct string_view *in)\n {\n \tint ptr = 0;\n@@ -926,58 +930,6 @@ static struct reftable_record_vtable reftable_log_record_vtable = {\n \t.is_deletion = &reftable_log_record_is_deletion_void,\n };\n \n-struct reftable_record reftable_new_record(uint8_t typ)\n-{\n-\tstruct reftable_record rec = { NULL };\n-\tswitch (typ) {\n-\tcase BLOCK_TYPE_REF: {\n-\t\tstruct reftable_ref_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_ref_record));\n-\t\treftable_record_from_ref(&rec, r);\n-\t\treturn rec;\n-\t}\n-\n-\tcase BLOCK_TYPE_OBJ: {\n-\t\tstruct reftable_obj_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_obj_record));\n-\t\treftable_record_from_obj(&rec, r);\n-\t\treturn rec;\n-\t}\n-\tcase BLOCK_TYPE_LOG: {\n-\t\tstruct reftable_log_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_log_record));\n-\t\treftable_record_from_log(&rec, r);\n-\t\treturn rec;\n-\t}\n-\tcase BLOCK_TYPE_INDEX: {\n-\t\tstruct reftable_index_record empty = { .last_key =\n-\t\t\t\t\t\t\t STRBUF_INIT };\n-\t\tstruct reftable_index_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_index_record));\n-\t\t*r = empty;\n-\t\treftable_record_from_index(&rec, r);\n-\t\treturn rec;\n-\t}\n-\t}\n-\tabort();\n-\treturn rec;\n-}\n-\n-/* clear out the record, yielding the reftable_record data that was\n- * encapsulated. */\n-static void *reftable_record_yield(struct reftable_record *rec)\n-{\n-\tvoid *p = rec->data;\n-\trec->data = NULL;\n-\treturn p;\n-}\n-\n-void reftable_record_destroy(struct reftable_record *rec)\n-{\n-\treftable_record_release(rec);\n-\treftable_free(reftable_record_yield(rec));\n-}\n-\n static void reftable_index_record_key(const void *r, struct strbuf *dest)\n {\n \tconst struct reftable_index_record *rec = r;\n@@ -1055,91 +1007,47 @@ static struct reftable_record_vtable reftable_index_record_vtable = {\n \n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)\n {\n-\trec->ops->key(rec->data, dest);\n-}\n-\n-uint8_t reftable_record_type(struct reftable_record *rec)\n-{\n-\treturn rec->ops->type;\n+\treftable_record_vtable(rec)->key(reftable_record_data(rec), dest);\n }\n \n int reftable_record_encode(struct reftable_record *rec, struct string_view dest,\n \t\t\t int hash_size)\n {\n-\treturn rec->ops->encode(rec->data, dest, hash_size);\n+\treturn reftable_record_vtable(rec)->encode(reftable_record_data(rec),\n+\t\t\t\t\t\t dest, hash_size);\n }\n \n void reftable_record_copy_from(struct reftable_record *rec,\n \t\t\t struct reftable_record *src, int hash_size)\n {\n-\tassert(src->ops->type == rec->ops->type);\n+\tassert(src->type == rec->type);\n \n-\trec->ops->copy_from(rec->data, src->data, hash_size);\n+\treftable_record_vtable(rec)->copy_from(reftable_record_data(rec),\n+\t\t\t\t\t reftable_record_data(src),\n+\t\t\t\t\t hash_size);\n }\n \n uint8_t reftable_record_val_type(struct reftable_record *rec)\n {\n-\treturn rec->ops->val_type(rec->data);\n+\treturn reftable_record_vtable(rec)->val_type(reftable_record_data(rec));\n }\n \n int reftable_record_decode(struct reftable_record *rec, struct strbuf key,\n \t\t\t uint8_t extra, struct string_view src, int hash_size)\n {\n-\treturn rec->ops->decode(rec->data, key, extra, src, hash_size);\n+\treturn reftable_record_vtable(rec)->decode(reftable_record_data(rec),\n+\t\t\t\t\t\t key, extra, src, hash_size);\n }\n \n void reftable_record_release(struct reftable_record *rec)\n {\n-\trec->ops->release(rec->data);\n+\treftable_record_vtable(rec)->release(reftable_record_data(rec));\n }\n \n int reftable_record_is_deletion(struct reftable_record *rec)\n {\n-\treturn rec->ops->is_deletion(rec->data);\n-}\n-\n-void reftable_record_from_ref(struct reftable_record *rec,\n-\t\t\t struct reftable_ref_record *ref_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = ref_rec;\n-\trec->ops = &reftable_ref_record_vtable;\n-}\n-\n-void reftable_record_from_obj(struct reftable_record *rec,\n-\t\t\t struct reftable_obj_record *obj_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = obj_rec;\n-\trec->ops = &reftable_obj_record_vtable;\n-}\n-\n-void reftable_record_from_index(struct reftable_record *rec,\n-\t\t\t\tstruct reftable_index_record *index_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = index_rec;\n-\trec->ops = &reftable_index_record_vtable;\n-}\n-\n-void reftable_record_from_log(struct reftable_record *rec,\n-\t\t\t struct reftable_log_record *log_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = log_rec;\n-\trec->ops = &reftable_log_record_vtable;\n-}\n-\n-struct reftable_ref_record *reftable_record_as_ref(struct reftable_record *rec)\n-{\n-\tassert(reftable_record_type(rec) == BLOCK_TYPE_REF);\n-\treturn rec->data;\n-}\n-\n-struct reftable_log_record *reftable_record_as_log(struct reftable_record *rec)\n-{\n-\tassert(reftable_record_type(rec) == BLOCK_TYPE_LOG);\n-\treturn rec->data;\n+\treturn reftable_record_vtable(rec)->is_deletion(\n+\t\treftable_record_data(rec));\n }\n \n static int hash_equal(uint8_t *a, uint8_t *b, int hash_size)\n@@ -1210,3 +1118,53 @@ void string_view_consume(struct string_view *s, int n)\n \ts->buf += n;\n \ts->len -= n;\n }\n+\n+static void *reftable_record_data(struct reftable_record *rec)\n+{\n+\tswitch (rec->type) {\n+\tcase BLOCK_TYPE_REF:\n+\t\treturn &rec->ref;\n+\tcase BLOCK_TYPE_LOG:\n+\t\treturn &rec->log;\n+\tcase BLOCK_TYPE_INDEX:\n+\t\treturn &rec->idx;\n+\tcase BLOCK_TYPE_OBJ:\n+\t\treturn &rec->obj;\n+\t}\n+\tabort();\n+}\n+\n+static struct reftable_record_vtable *\n+reftable_record_vtable(struct reftable_record *rec)\n+{\n+\tswitch (rec->type) {\n+\tcase BLOCK_TYPE_REF:\n+\t\treturn &reftable_ref_record_vtable;\n+\tcase BLOCK_TYPE_LOG:\n+\t\treturn &reftable_log_record_vtable;\n+\tcase BLOCK_TYPE_INDEX:\n+\t\treturn &reftable_index_record_vtable;\n+\tcase BLOCK_TYPE_OBJ:\n+\t\treturn &reftable_obj_record_vtable;\n+\t}\n+\tabort();\n+}\n+\n+struct reftable_record reftable_record_for(uint8_t typ)\n+{\n+\tstruct reftable_record clean_idx = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.idx = {\n+\t\t\t.last_key = STRBUF_INIT,\n+\t\t},\n+\t};\n+\tstruct reftable_record clean = {\n+\t\t.type = typ,\n+\t};\n+\n+\tif (typ == BLOCK_TYPE_INDEX) {\n+\t\treturn clean_idx;\n+\t}\n+\n+\treturn clean;\n+}\ndiff --git a/reftable/record.h b/reftable/record.h\nindex 498e8c50bf4..a702c67e857 100644\n--- a/reftable/record.h\n+++ b/reftable/record.h\n@@ -60,18 +60,9 @@ struct reftable_record_vtable {\n \tint (*is_deletion)(const void *rec);\n };\n \n-/* record is a generic wrapper for different types of records. */\n-struct reftable_record {\n-\tvoid *data;\n-\tstruct reftable_record_vtable *ops;\n-};\n-\n /* returns true for recognized block types. Block start with the block type. */\n int reftable_is_block_type(uint8_t typ);\n \n-/* creates a malloced record of the given type. Dispose with record_destroy */\n-struct reftable_record reftable_new_record(uint8_t typ);\n-\n /* Encode `key` into `dest`. Sets `is_restart` to indicate a restart. Returns\n * number of bytes written. */\n int reftable_encode_key(int *is_restart, struct string_view dest,\n@@ -97,8 +88,26 @@ struct reftable_obj_record {\n \tint offset_len;\n };\n \n+/* record is a generic wrapper for different types of records. It is normally\n+ * created on the stack, or embedded within another struct. If the type is\n+ * known, a fresh instance can be initialized explicitly. Otherwise, use\n+ * reftable_record_for() to initialize generically (as the index_record is not\n+ * valid as 0-initialized structure)\n+ */\n+struct reftable_record {\n+\tuint8_t type;\n+\tunion {\n+\t\tstruct reftable_ref_record ref;\n+\t\tstruct reftable_log_record log;\n+\t\tstruct reftable_obj_record obj;\n+\t\tstruct reftable_index_record idx;\n+\t};\n+};\n+\n /* see struct record_vtable */\n \n+/* return an initialized record for the given type */\n+struct reftable_record reftable_record_for(uint8_t typ);\n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest);\n uint8_t reftable_record_type(struct reftable_record *rec);\n void reftable_record_copy_from(struct reftable_record *rec,\n@@ -111,25 +120,9 @@ int reftable_record_decode(struct reftable_record *rec, struct strbuf key,\n \t\t\t int hash_size);\n int reftable_record_is_deletion(struct reftable_record *rec);\n \n-/* zeroes out the embedded record */\n+/* frees and zeroes out the embedded record */\n void reftable_record_release(struct reftable_record *rec);\n \n-/* clear and deallocate embedded record, and zero `rec`. */\n-void reftable_record_destroy(struct reftable_record *rec);\n-\n-/* initialize generic records from concrete records. The generic record should\n- * be zeroed out. */\n-void reftable_record_from_obj(struct reftable_record *rec,\n-\t\t\t struct reftable_obj_record *objrec);\n-void reftable_record_from_index(struct reftable_record *rec,\n-\t\t\t\tstruct reftable_index_record *idxrec);\n-void reftable_record_from_ref(struct reftable_record *rec,\n-\t\t\t struct reftable_ref_record *refrec);\n-void reftable_record_from_log(struct reftable_record *rec,\n-\t\t\t struct reftable_log_record *logrec);\n-struct reftable_ref_record *reftable_record_as_ref(struct reftable_record *ref);\n-struct reftable_log_record *reftable_record_as_log(struct reftable_record *ref);\n-\n /* for qsort. */\n int reftable_ref_record_compare_name(const void *a, const void *b);\n \ndiff --git a/reftable/record_test.c b/reftable/record_test.c\nindex f4ad7cace41..23abfb037b3 100644\n--- a/reftable/record_test.c\n+++ b/reftable/record_test.c\n@@ -16,24 +16,22 @@\n \n static void test_copy(struct reftable_record *rec)\n {\n-\tstruct reftable_record copy =\n-\t\treftable_new_record(reftable_record_type(rec));\n+\tstruct reftable_record copy = reftable_record_for(rec->type);\n+\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \t/* do it twice to catch memory leaks */\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n-\tswitch (reftable_record_type(©)) {\n+\tswitch (copy.type) {\n \tcase BLOCK_TYPE_REF:\n-\t\tEXPECT(reftable_ref_record_equal(reftable_record_as_ref(©),\n-\t\t\t\t\t\t reftable_record_as_ref(rec),\n+\t\tEXPECT(reftable_ref_record_equal(©.ref, &rec->ref,\n \t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\tbreak;\n \tcase BLOCK_TYPE_LOG:\n-\t\tEXPECT(reftable_log_record_equal(reftable_record_as_log(©),\n-\t\t\t\t\t\t reftable_record_as_log(rec),\n+\t\tEXPECT(reftable_log_record_equal(©.log, &rec->log,\n \t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\tbreak;\n \t}\n-\treftable_record_destroy(©);\n+\treftable_record_release(©);\n }\n \n static void test_varint_roundtrip(void)\n@@ -106,61 +104,60 @@ static void test_reftable_ref_record_roundtrip(void)\n \tint i = 0;\n \n \tfor (i = REFTABLE_REF_DELETION; i < REFTABLE_NR_REF_VALUETYPES; i++) {\n-\t\tstruct reftable_ref_record in = { NULL };\n-\t\tstruct reftable_ref_record out = { NULL };\n-\t\tstruct reftable_record rec_out = { NULL };\n+\t\tstruct reftable_record in = {\n+\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t};\n+\t\tstruct reftable_record out = {\n+\t\t\t.type = BLOCK_TYPE_REF\n+\t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n-\t\tstruct reftable_record rec = { NULL };\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n \t\t\t.buf = buffer,\n \t\t\t.len = sizeof(buffer),\n \t\t};\n-\n \t\tint n, m;\n \n-\t\tin.value_type = i;\n+\t\tin.ref.value_type = i;\n \t\tswitch (i) {\n \t\tcase REFTABLE_REF_DELETION:\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_VAL1:\n-\t\t\tin.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val1, 1);\n+\t\t\tin.ref.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n+\t\t\tset_hash(in.ref.value.val1, 1);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_VAL2:\n-\t\t\tin.value.val2.value = reftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val2.value, 1);\n-\t\t\tin.value.val2.target_value =\n+\t\t\tin.ref.value.val2.value =\n+\t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n+\t\t\tset_hash(in.ref.value.val2.value, 1);\n+\t\t\tin.ref.value.val2.target_value =\n \t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val2.target_value, 2);\n+\t\t\tset_hash(in.ref.value.val2.target_value, 2);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_SYMREF:\n-\t\t\tin.value.symref = xstrdup(\"target\");\n+\t\t\tin.ref.value.symref = xstrdup(\"target\");\n \t\t\tbreak;\n \t\t}\n-\t\tin.refname = xstrdup(\"refs/heads/master\");\n+\t\tin.ref.refname = xstrdup(\"refs/heads/master\");\n \n-\t\treftable_record_from_ref(&rec, &in);\n-\t\ttest_copy(&rec);\n+\t\ttest_copy(&in);\n \n-\t\tEXPECT(reftable_record_val_type(&rec) == i);\n+\t\tEXPECT(reftable_record_val_type(&in) == i);\n \n-\t\treftable_record_key(&rec, &key);\n-\t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\t\treftable_record_key(&in, &key);\n+\t\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n > 0);\n \n \t\t/* decode into a non-zero reftable_record to test for leaks. */\n-\n-\t\treftable_record_from_ref(&rec_out, &out);\n-\t\tm = reftable_record_decode(&rec_out, key, i, dest,\n-\t\t\t\t\t GIT_SHA1_RAWSZ);\n+\t\tm = reftable_record_decode(&out, key, i, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_ref_record_equal(&in, &out, GIT_SHA1_RAWSZ));\n-\t\treftable_record_release(&rec_out);\n+\t\tEXPECT(reftable_ref_record_equal(&in.ref, &out.ref,\n+\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n+\t\treftable_record_release(&in);\n \n \t\tstrbuf_release(&key);\n-\t\treftable_ref_record_release(&in);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n@@ -213,7 +210,9 @@ static void test_reftable_log_record_roundtrip(void)\n \tset_test_hash(in[0].value.update.new_hash, 1);\n \tset_test_hash(in[0].value.update.old_hash, 2);\n \tfor (i = 0; i < ARRAY_SIZE(in); i++) {\n-\t\tstruct reftable_record rec = { NULL };\n+\t\tstruct reftable_record rec = {\n+\t\t\t.type = BLOCK_TYPE_LOG\n+\t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n@@ -221,8 +220,10 @@ static void test_reftable_log_record_roundtrip(void)\n \t\t\t.len = sizeof(buffer),\n \t\t};\n \t\t/* populate out, to check for leaks. */\n-\t\tstruct reftable_log_record out = {\n-\t\t\t.refname = xstrdup(\"old name\"),\n+\t\tstruct reftable_record out = {\n+\t\t\t.type = BLOCK_TYPE_LOG,\n+\t\t\t.log = {\n+\t\t\t\t.refname = xstrdup(\"old name\"),\n \t\t\t.value_type = REFTABLE_LOG_UPDATE,\n \t\t\t.value = {\n \t\t\t\t.update = {\n@@ -233,11 +234,11 @@ static void test_reftable_log_record_roundtrip(void)\n \t\t\t\t\t.message = xstrdup(\"old message\"),\n \t\t\t\t},\n \t\t\t},\n+\t\t\t},\n \t\t};\n-\t\tstruct reftable_record rec_out = { NULL };\n \t\tint n, m, valtype;\n \n-\t\treftable_record_from_log(&rec, &in[i]);\n+\t\trec.log = in[i];\n \n \t\ttest_copy(&rec);\n \n@@ -245,16 +246,16 @@ static void test_reftable_log_record_roundtrip(void)\n \n \t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n >= 0);\n-\t\treftable_record_from_log(&rec_out, &out);\n \t\tvaltype = reftable_record_val_type(&rec);\n-\t\tm = reftable_record_decode(&rec_out, key, valtype, dest,\n+\t\tm = reftable_record_decode(&out, key, valtype, dest,\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_log_record_equal(&in[i], &out, GIT_SHA1_RAWSZ));\n+\t\tEXPECT(reftable_log_record_equal(&in[i], &out.log,\n+\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\treftable_log_record_release(&in[i]);\n \t\tstrbuf_release(&key);\n-\t\treftable_record_release(&rec_out);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n@@ -322,47 +323,51 @@ static void test_reftable_obj_record_roundtrip(void)\n \t\t\t\t\t } };\n \tint i = 0;\n \tfor (i = 0; i < ARRAY_SIZE(recs); i++) {\n-\t\tstruct reftable_obj_record in = recs[i];\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n \t\t\t.buf = buffer,\n \t\t\t.len = sizeof(buffer),\n \t\t};\n-\t\tstruct reftable_record rec = { NULL };\n+\t\tstruct reftable_record in = {\n+\t\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t\t.obj = recs[i],\n+\t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n-\t\tstruct reftable_obj_record out = { NULL };\n-\t\tstruct reftable_record rec_out = { NULL };\n+\t\tstruct reftable_record out = {\n+\t\t\t.type = BLOCK_TYPE_OBJ\n+\t\t};\n \t\tint n, m;\n \t\tuint8_t extra;\n \n-\t\treftable_record_from_obj(&rec, &in);\n-\t\ttest_copy(&rec);\n-\t\treftable_record_key(&rec, &key);\n-\t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\t\ttest_copy(&in);\n+\t\treftable_record_key(&in, &key);\n+\t\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n > 0);\n-\t\textra = reftable_record_val_type(&rec);\n-\t\treftable_record_from_obj(&rec_out, &out);\n-\t\tm = reftable_record_decode(&rec_out, key, extra, dest,\n+\t\textra = reftable_record_val_type(&in);\n+\t\tm = reftable_record_decode(&out, key, extra, dest,\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(in.hash_prefix_len == out.hash_prefix_len);\n-\t\tEXPECT(in.offset_len == out.offset_len);\n+\t\tEXPECT(in.obj.hash_prefix_len == out.obj.hash_prefix_len);\n+\t\tEXPECT(in.obj.offset_len == out.obj.offset_len);\n \n-\t\tEXPECT(!memcmp(in.hash_prefix, out.hash_prefix,\n-\t\t\t in.hash_prefix_len));\n-\t\tEXPECT(0 == memcmp(in.offsets, out.offsets,\n-\t\t\t\t sizeof(uint64_t) * in.offset_len));\n+\t\tEXPECT(!memcmp(in.obj.hash_prefix, out.obj.hash_prefix,\n+\t\t\t in.obj.hash_prefix_len));\n+\t\tEXPECT(0 == memcmp(in.obj.offsets, out.obj.offsets,\n+\t\t\t\t sizeof(uint64_t) * in.obj.offset_len));\n \t\tstrbuf_release(&key);\n-\t\treftable_record_release(&rec_out);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n static void test_reftable_index_record_roundtrip(void)\n {\n-\tstruct reftable_index_record in = {\n-\t\t.offset = 42,\n-\t\t.last_key = STRBUF_INIT,\n+\tstruct reftable_record in = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.idx = {\n+\t\t\t.offset = 42,\n+\t\t\t.last_key = STRBUF_INIT,\n+\t\t},\n \t};\n \tuint8_t buffer[1024] = { 0 };\n \tstruct string_view dest = {\n@@ -370,31 +375,30 @@ static void test_reftable_index_record_roundtrip(void)\n \t\t.len = sizeof(buffer),\n \t};\n \tstruct strbuf key = STRBUF_INIT;\n-\tstruct reftable_record rec = { NULL };\n-\tstruct reftable_index_record out = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record out_rec = { NULL };\n+\tstruct reftable_record out = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.idx = { .last_key = STRBUF_INIT },\n+\t};\n \tint n, m;\n \tuint8_t extra;\n \n-\tstrbuf_addstr(&in.last_key, \"refs/heads/master\");\n-\treftable_record_from_index(&rec, &in);\n-\treftable_record_key(&rec, &key);\n-\ttest_copy(&rec);\n+\tstrbuf_addstr(&in.idx.last_key, \"refs/heads/master\");\n+\treftable_record_key(&in, &key);\n+\ttest_copy(&in);\n \n-\tEXPECT(0 == strbuf_cmp(&key, &in.last_key));\n-\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\tEXPECT(0 == strbuf_cmp(&key, &in.idx.last_key));\n+\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(n > 0);\n \n-\textra = reftable_record_val_type(&rec);\n-\treftable_record_from_index(&out_rec, &out);\n-\tm = reftable_record_decode(&out_rec, key, extra, dest, GIT_SHA1_RAWSZ);\n+\textra = reftable_record_val_type(&in);\n+\tm = reftable_record_decode(&out, key, extra, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(m == n);\n \n-\tEXPECT(in.offset == out.offset);\n+\tEXPECT(in.idx.offset == out.idx.offset);\n \n-\treftable_record_release(&out_rec);\n+\treftable_record_release(&out);\n \tstrbuf_release(&key);\n-\tstrbuf_release(&in.last_key);\n+\tstrbuf_release(&in.idx.last_key);\n }\n \n int record_test_main(int argc, const char *argv[])\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex 3ca721e9f64..df256f3a8b5 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -221,10 +221,10 @@ static int writer_add_record(struct reftable_writer *w,\n \tstrbuf_reset(&w->last_key);\n \tstrbuf_addbuf(&w->last_key, &key);\n \tif (w->block_writer == NULL) {\n-\t\twriter_reinit_block_writer(w, reftable_record_type(rec));\n+\t\twriter_reinit_block_writer(w, rec->type);\n \t}\n \n-\tassert(block_writer_type(w->block_writer) == reftable_record_type(rec));\n+\tassert(block_writer_type(w->block_writer) == rec->type);\n \n \tif (block_writer_add(w->block_writer, rec) == 0) {\n \t\terr = 0;\n@@ -236,7 +236,7 @@ static int writer_add_record(struct reftable_writer *w,\n \t\tgoto done;\n \t}\n \n-\twriter_reinit_block_writer(w, reftable_record_type(rec));\n+\twriter_reinit_block_writer(w, rec->type);\n \terr = block_writer_add(w->block_writer, rec);\n \tif (err < 0) {\n \t\tgoto done;\n@@ -251,8 +251,10 @@ done:\n int reftable_writer_add_ref(struct reftable_writer *w,\n \t\t\t struct reftable_ref_record *ref)\n {\n-\tstruct reftable_record rec = { NULL };\n-\tstruct reftable_ref_record copy = *ref;\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.ref = *ref,\n+\t};\n \tint err = 0;\n \n \tif (ref->refname == NULL)\n@@ -261,8 +263,7 @@ int reftable_writer_add_ref(struct reftable_writer *w,\n \t ref->update_index > w->max_update_index)\n \t\treturn REFTABLE_API_ERROR;\n \n-\treftable_record_from_ref(&rec, ©);\n-\tcopy.update_index -= w->min_update_index;\n+\trec.ref.update_index -= w->min_update_index;\n \n \terr = writer_add_record(w, &rec);\n \tif (err < 0)\n@@ -301,7 +302,10 @@ int reftable_writer_add_refs(struct reftable_writer *w,\n static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n \t\t\t\t\t struct reftable_log_record *log)\n {\n-\tstruct reftable_record rec = { NULL };\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.log = *log,\n+\t};\n \tif (w->block_writer &&\n \t block_writer_type(w->block_writer) == BLOCK_TYPE_REF) {\n \t\tint err = writer_finish_public_section(w);\n@@ -311,8 +315,6 @@ static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n \n \tw->next -= w->pending_padding;\n \tw->pending_padding = 0;\n-\n-\treftable_record_from_log(&rec, log);\n \treturn writer_add_record(w, &rec);\n }\n \n@@ -393,8 +395,10 @@ static int writer_finish_section(struct reftable_writer *w)\n \t\tw->index_len = 0;\n \t\tw->index_cap = 0;\n \t\tfor (i = 0; i < idx_len; i++) {\n-\t\t\tstruct reftable_record rec = { NULL };\n-\t\t\treftable_record_from_index(&rec, idx + i);\n+\t\t\tstruct reftable_record rec = {\n+\t\t\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t\t\t.idx = idx[i],\n+\t\t\t};\n \t\t\tif (block_writer_add(w->block_writer, &rec) == 0) {\n \t\t\t\tcontinue;\n \t\t\t}\n@@ -462,17 +466,18 @@ static void write_object_record(void *void_arg, void *key)\n {\n \tstruct write_record_arg *arg = void_arg;\n \tstruct obj_index_tree_node *entry = key;\n-\tstruct reftable_obj_record obj_rec = {\n-\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n-\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n-\t\t.offsets = entry->offsets,\n-\t\t.offset_len = entry->offset_len,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.obj = {\n+\t\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n+\t\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n+\t\t\t.offsets = entry->offsets,\n+\t\t\t.offset_len = entry->offset_len,\n+\t\t}\n \t};\n-\tstruct reftable_record rec = { NULL };\n \tif (arg->err < 0)\n \t\tgoto done;\n \n-\treftable_record_from_obj(&rec, &obj_rec);\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \tif (arg->err == 0)\n \t\tgoto done;\n@@ -485,7 +490,8 @@ static void write_object_record(void *void_arg, void *key)\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \tif (arg->err == 0)\n \t\tgoto done;\n-\tobj_rec.offset_len = 0;\n+\n+\trec.obj.offset_len = 0;\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \n \t/* Should be able to write into a fresh block. */\n-- \ngitgitgadget\n"},{"id":"443378","messageId":"xmqqlf0w5bbc.fsf@gitster.g","threadId":"57047","inReplyTo":"8deccc3a1dff7e4f7d613fa63d2781fd1f11f841.1638899124.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 10/10] reftable: make reftable_record a tagged union","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-07T21:56:07Z","receivedAt":"2021-12-07T21:56:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Han-Wen Nienhuys via GitGitGadget\" writes:\n\n> +struct reftable_record {\n> +\tuint8_t type;\n> +\tunion {\n> +\t\tstruct reftable_ref_record ref;\n> +\t\tstruct reftable_log_record log;\n> +\t\tstruct reftable_obj_record obj;\n> +\t\tstruct reftable_index_record idx;\n> +\t};\n> +};\n\nerror: ISO C99 doesn't support unnamed structs/unions [-Werror=pedantic]\n"},{"id":"443399","messageId":"YbAVOtYXA1Hf9EtJ@coredump.intra.peff.net","threadId":"57047","inReplyTo":"xmqqlf0w5bbc.fsf@gitster.g","subject":"Re: [PATCH 10/10] reftable: make reftable_record a tagged union","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-12-08T02:15:22Z","receivedAt":"2021-12-08T02:15:24Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Dec 07, 2021 at 01:56:07PM -0800, Junio C Hamano wrote:\n\n> \"Han-Wen Nienhuys via GitGitGadget\" writes:\n> \n> > +struct reftable_record {\n> > +\tuint8_t type;\n> > +\tunion {\n> > +\t\tstruct reftable_ref_record ref;\n> > +\t\tstruct reftable_log_record log;\n> > +\t\tstruct reftable_obj_record obj;\n> > +\t\tstruct reftable_index_record idx;\n> > +\t};\n> > +};\n> \n> error: ISO C99 doesn't support unnamed structs/unions [-Werror=pedantic]\n\nHmm. It's interesting that the regular DEVELOPER=1 doesn't catch this.\nIt's because we don't specify -std there, and newer gcc defaults to\ngnu17 (unnamed unions appeared in c11, I think). I wonder if it would be\nhelpful to teach config.mak.dev to pass -std=c99.\n\nAnyway, the usual fix is to stick a \"u.foo\" everywhere. Patch is below\nfor convenience. Two things to note:\n\n - I used \".u.foo = {...}\" as a designated initializer. I think this is\n allowed by the standard, but I don't think it's something we've used\n before. If it's a problem, it can be solved by using an extra level\n of braces (which is OK, just uglier).\n\n - Han-Wen, if you don't just squash in the patch below, note that it\n also has a fixup for some misleading indentation in an initializer\n in test_reftable_log_record_roundtrip(). You might want to grab that\n separately.\n\n---\n reftable/block_test.c | 16 ++++----\n reftable/generic.c | 8 ++--\n reftable/iter.c | 4 +-\n reftable/merged.c | 4 +-\n reftable/pq_test.c | 8 ++--\n reftable/reader.c | 26 ++++++------\n reftable/record.c | 10 ++---\n reftable/record.h | 2 +-\n reftable/record_test.c | 74 +++++++++++++++++------------------\n reftable/writer.c | 12 +++---\n 10 files changed, 82 insertions(+), 82 deletions(-)\n\ndiff --git a/reftable/block_test.c b/reftable/block_test.c\nindex 670329f222..fa2ee092ec 100644\n--- a/reftable/block_test.c\n+++ b/reftable/block_test.c\n@@ -48,14 +48,14 @@ static void test_block_read_write(void)\n \t\tsnprintf(name, sizeof(name), \"branch%02d\", i);\n \t\tmemset(hash, i, sizeof(hash));\n \n-\t\trec.ref.refname = name;\n-\t\trec.ref.value_type = REFTABLE_REF_VAL1;\n-\t\trec.ref.value.val1 = hash;\n+\t\trec.u.ref.refname = name;\n+\t\trec.u.ref.value_type = REFTABLE_REF_VAL1;\n+\t\trec.u.ref.value.val1 = hash;\n \n \t\tnames[i] = xstrdup(name);\n \t\tn = block_writer_add(&bw, &rec);\n-\t\trec.ref.refname = NULL;\n-\t\trec.ref.value_type = REFTABLE_REF_DELETION;\n+\t\trec.u.ref.refname = NULL;\n+\t\trec.u.ref.value_type = REFTABLE_REF_DELETION;\n \t\tEXPECT(n == 0);\n \t}\n \n@@ -74,7 +74,7 @@ static void test_block_read_write(void)\n \t\tif (r > 0) {\n \t\t\tbreak;\n \t\t}\n-\t\tEXPECT_STREQ(names[j], rec.ref.refname);\n+\t\tEXPECT_STREQ(names[j], rec.u.ref.refname);\n \t\tj++;\n \t}\n \n@@ -92,15 +92,15 @@ static void test_block_read_write(void)\n \t\tn = block_iter_next(&it, &rec);\n \t\tEXPECT(n == 0);\n \n-\t\tEXPECT_STREQ(names[i], rec.ref.refname);\n+\t\tEXPECT_STREQ(names[i], rec.u.ref.refname);\n \n \t\twant.len--;\n \t\tn = block_reader_seek(&br, &it, &want);\n \t\tEXPECT(n == 0);\n \n \t\tn = block_iter_next(&it, &rec);\n \t\tEXPECT(n == 0);\n-\t\tEXPECT_STREQ(names[10 * (i / 10)], rec.ref.refname);\n+\t\tEXPECT_STREQ(names[10 * (i / 10)], rec.u.ref.refname);\n \n \t\tblock_iter_close(&it);\n \t}\ndiff --git a/reftable/generic.c b/reftable/generic.c\nindex 7f00aa9cdb..4446b8ed36 100644\n--- a/reftable/generic.c\n+++ b/reftable/generic.c\n@@ -17,7 +17,7 @@ int reftable_table_seek_ref(struct reftable_table *tab,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n \tstruct reftable_record rec = { .type = BLOCK_TYPE_REF,\n-\t\t\t\t .ref = {\n+\t\t\t\t .u.ref = {\n \t\t\t\t\t .refname = (char *)name,\n \t\t\t\t } };\n \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n@@ -27,7 +27,7 @@ int reftable_table_seek_log(struct reftable_table *tab,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n \tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n-\t\t\t\t .log = {\n+\t\t\t\t .u.log = {\n \t\t\t\t\t .refname = (char *)name,\n \t\t\t\t\t .update_index = ~((uint64_t)0),\n \t\t\t\t } };\n@@ -130,7 +130,7 @@ int reftable_iterator_next_ref(struct reftable_iterator *it,\n {\n \tstruct reftable_record rec = { .type = BLOCK_TYPE_REF };\n \tint err = iterator_next(it, &rec);\n-\t*ref = rec.ref;\n+\t*ref = rec.u.ref;\n \treturn err;\n }\n \n@@ -139,7 +139,7 @@ int reftable_iterator_next_log(struct reftable_iterator *it,\n {\n \tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG };\n \tint err = iterator_next(it, &rec);\n-\t*log = rec.log;\n+\t*log = rec.u.log;\n \treturn err;\n }\n \ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex fedf5d8ab0..a8d174c040 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -32,7 +32,7 @@ static int filtering_ref_iterator_next(void *iter_arg,\n \t\t\t\t struct reftable_record *rec)\n {\n \tstruct filtering_ref_iterator *fri = iter_arg;\n-\tstruct reftable_ref_record *ref = &rec->ref;\n+\tstruct reftable_ref_record *ref = &rec->u.ref;\n \tint err = 0;\n \twhile (1) {\n \t\terr = reftable_iterator_next_ref(&fri->it, ref);\n@@ -127,7 +127,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n static int indexed_table_ref_iter_next(void *p, struct reftable_record *rec)\n {\n \tstruct indexed_table_ref_iter *it = p;\n-\tstruct reftable_ref_record *ref = &rec->ref;\n+\tstruct reftable_ref_record *ref = &rec->u.ref;\n \n \twhile (1) {\n \t\tint err = block_iter_next(&it->cur, rec);\ndiff --git a/reftable/merged.c b/reftable/merged.c\nindex e69955d7c2..e9b0edec97 100644\n--- a/reftable/merged.c\n+++ b/reftable/merged.c\n@@ -291,7 +291,7 @@ int reftable_merged_table_seek_ref(struct reftable_merged_table *mt,\n {\n \tstruct reftable_record rec = {\n \t\t.type = BLOCK_TYPE_REF,\n-\t\t.ref = {\n+\t\t.u.ref = {\n \t\t\t.refname = (char *)name,\n \t\t},\n \t};\n@@ -304,7 +304,7 @@ int reftable_merged_table_seek_log_at(struct reftable_merged_table *mt,\n {\n \tstruct reftable_record rec = {\n \t\t.type = BLOCK_TYPE_LOG,\n-\t\t.log = {\n+\t\t.u.log = {\n \t\t\t.refname = (char *)name,\n \t\t\t.update_index = update_index,\n \t\t}\ndiff --git a/reftable/pq_test.c b/reftable/pq_test.c\nindex 89d069d674..9f04d84fed 100644\n--- a/reftable/pq_test.c\n+++ b/reftable/pq_test.c\n@@ -45,7 +45,7 @@ static void test_pq(void)\n \t\tstruct pq_entry e = {\n \t\t\t.rec = {\n \t\t\t\t.type = BLOCK_TYPE_REF,\n-\t\t\t\t.ref = {\n+\t\t\t\t.u.ref = {\n \t\t\t\t\t.refname = names[i],\n \t\t\t\t}\n \t\t\t}\n@@ -62,11 +62,11 @@ static void test_pq(void)\n \n \t\tEXPECT(rec->type == BLOCK_TYPE_REF);\n \t\tif (last) {\n-\t\t\tEXPECT(strcmp(last, rec->ref.refname) < 0);\n+\t\t\tEXPECT(strcmp(last, rec->u.ref.refname) < 0);\n \t\t}\n \t\t// this is names[i], so don't dealloc.\n-\t\tlast = rec->ref.refname;\n-\t\trec->ref.refname = NULL;\n+\t\tlast = rec->u.ref.refname;\n+\t\trec->u.ref.refname = NULL;\n \t\treftable_record_release(rec);\n \t}\n \tfor (i = 0; i < N; i++) {\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 0159f13112..ef322791fd 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -239,7 +239,7 @@ static int table_iter_next_in_block(struct table_iter *ti,\n {\n \tint res = block_iter_next(&ti->bi, rec);\n \tif (res == 0 && rec->type == BLOCK_TYPE_REF) {\n-\t\trec->ref.update_index += ti->r->min_update_index;\n+\t\trec->u.ref.update_index += ti->r->min_update_index;\n \t}\n \n \treturn res;\n@@ -485,17 +485,17 @@ static int reader_seek_indexed(struct reftable_reader *r,\n {\n \tstruct reftable_record want_index = {\n \t\t.type = BLOCK_TYPE_INDEX,\n-\t\t.idx = { .last_key = STRBUF_INIT }\n+\t\t.u.idx = { .last_key = STRBUF_INIT }\n \t};\n \tstruct reftable_record index_result = {\n \t\t.type = BLOCK_TYPE_INDEX,\n-\t\t.idx = { .last_key = STRBUF_INIT },\n+\t\t.u.idx = { .last_key = STRBUF_INIT },\n \t};\n \tstruct table_iter index_iter = TABLE_ITER_INIT;\n \tstruct table_iter next = TABLE_ITER_INIT;\n \tint err = 0;\n \n-\treftable_record_key(rec, &want_index.idx.last_key);\n+\treftable_record_key(rec, &want_index.u.idx.last_key);\n \terr = reader_start(r, &index_iter, rec->type, 1);\n \tif (err < 0)\n \t\tgoto done;\n@@ -507,12 +507,12 @@ static int reader_seek_indexed(struct reftable_reader *r,\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = reader_table_iter_at(r, &next, index_result.idx.offset,\n+\t\terr = reader_table_iter_at(r, &next, index_result.u.idx.offset,\n \t\t\t\t\t 0);\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = block_iter_seek(&next.bi, &want_index.idx.last_key);\n+\t\terr = block_iter_seek(&next.bi, &want_index.u.idx.last_key);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n@@ -591,7 +591,7 @@ int reftable_reader_seek_ref(struct reftable_reader *r,\n {\n \tstruct reftable_record rec = {\n \t\t.type = BLOCK_TYPE_REF,\n-\t\t.ref = {\n+\t\t.u.ref = {\n \t\t\t.refname = (char *)name,\n \t\t},\n \t};\n@@ -604,7 +604,7 @@ int reftable_reader_seek_log_at(struct reftable_reader *r,\n {\n \tstruct reftable_record rec = {\n \t\t.type = BLOCK_TYPE_LOG,\n-\t\t.log = {\n+\t\t.u.log = {\n \t\t\t.refname = (char *)name,\n \t\t\t.update_index = update_index,\n \t\t}\n@@ -652,15 +652,15 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n {\n \tstruct reftable_record want = {\n \t\t.type = BLOCK_TYPE_OBJ,\n-\t\t.obj = {\n+\t\t.u.obj = {\n \t\t\t.hash_prefix = oid,\n \t\t\t.hash_prefix_len = r->object_id_len,\n \t\t},\n \t};\n \tstruct reftable_iterator oit = { NULL };\n \tstruct reftable_record got = {\n \t\t.type = BLOCK_TYPE_OBJ,\n-\t\t.obj = { 0 },\n+\t\t.u.obj = { 0 },\n \t};\n \tint err = 0;\n \tstruct indexed_table_ref_iter *itr = NULL;\n@@ -675,7 +675,7 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \tif (err < 0)\n \t\tgoto done;\n \n-\tif (err > 0 || memcmp(want.obj.hash_prefix, got.obj.hash_prefix,\n+\tif (err > 0 || memcmp(want.u.obj.hash_prefix, got.u.obj.hash_prefix,\n \t\t\t r->object_id_len)) {\n \t\t/* didn't find it; return empty iterator */\n \t\titerator_set_empty(it);\n@@ -684,10 +684,10 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \t}\n \n \terr = new_indexed_table_ref_iter(&itr, r, oid, hash_size(r->hash_id),\n-\t\t\t\t\t got.obj.offsets, got.obj.offset_len);\n+\t\t\t\t\t got.u.obj.offsets, got.u.obj.offset_len);\n \tif (err < 0)\n \t\tgoto done;\n-\tgot.obj.offsets = NULL;\n+\tgot.u.obj.offsets = NULL;\n \titerator_from_indexed_table_ref_iter(it, itr);\n \n done:\ndiff --git a/reftable/record.c b/reftable/record.c\nindex b665feb709..68fdde9d7c 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -1123,13 +1123,13 @@ static void *reftable_record_data(struct reftable_record *rec)\n {\n \tswitch (rec->type) {\n \tcase BLOCK_TYPE_REF:\n-\t\treturn &rec->ref;\n+\t\treturn &rec->u.ref;\n \tcase BLOCK_TYPE_LOG:\n-\t\treturn &rec->log;\n+\t\treturn &rec->u.log;\n \tcase BLOCK_TYPE_INDEX:\n-\t\treturn &rec->idx;\n+\t\treturn &rec->u.idx;\n \tcase BLOCK_TYPE_OBJ:\n-\t\treturn &rec->obj;\n+\t\treturn &rec->u.obj;\n \t}\n \tabort();\n }\n@@ -1154,7 +1154,7 @@ struct reftable_record reftable_record_for(uint8_t typ)\n {\n \tstruct reftable_record clean_idx = {\n \t\t.type = BLOCK_TYPE_INDEX,\n-\t\t.idx = {\n+\t\t.u.idx = {\n \t\t\t.last_key = STRBUF_INIT,\n \t\t},\n \t};\ndiff --git a/reftable/record.h b/reftable/record.h\nindex a702c67e85..1fe0c14a19 100644\n--- a/reftable/record.h\n+++ b/reftable/record.h\n@@ -101,7 +101,7 @@ struct reftable_record {\n \t\tstruct reftable_log_record log;\n \t\tstruct reftable_obj_record obj;\n \t\tstruct reftable_index_record idx;\n-\t};\n+\t} u;\n };\n \n /* see struct record_vtable */\ndiff --git a/reftable/record_test.c b/reftable/record_test.c\nindex 23abfb037b..f234a0382a 100644\n--- a/reftable/record_test.c\n+++ b/reftable/record_test.c\n@@ -23,11 +23,11 @@ static void test_copy(struct reftable_record *rec)\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \tswitch (copy.type) {\n \tcase BLOCK_TYPE_REF:\n-\t\tEXPECT(reftable_ref_record_equal(©.ref, &rec->ref,\n+\t\tEXPECT(reftable_ref_record_equal(©.u.ref, &rec->u.ref,\n \t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\tbreak;\n \tcase BLOCK_TYPE_LOG:\n-\t\tEXPECT(reftable_log_record_equal(©.log, &rec->log,\n+\t\tEXPECT(reftable_log_record_equal(©.u.log, &rec->u.log,\n \t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\tbreak;\n \t}\n@@ -118,27 +118,27 @@ static void test_reftable_ref_record_roundtrip(void)\n \t\t};\n \t\tint n, m;\n \n-\t\tin.ref.value_type = i;\n+\t\tin.u.ref.value_type = i;\n \t\tswitch (i) {\n \t\tcase REFTABLE_REF_DELETION:\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_VAL1:\n-\t\t\tin.ref.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.ref.value.val1, 1);\n+\t\t\tin.u.ref.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n+\t\t\tset_hash(in.u.ref.value.val1, 1);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_VAL2:\n-\t\t\tin.ref.value.val2.value =\n+\t\t\tin.u.ref.value.val2.value =\n \t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.ref.value.val2.value, 1);\n-\t\t\tin.ref.value.val2.target_value =\n+\t\t\tset_hash(in.u.ref.value.val2.value, 1);\n+\t\t\tin.u.ref.value.val2.target_value =\n \t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.ref.value.val2.target_value, 2);\n+\t\t\tset_hash(in.u.ref.value.val2.target_value, 2);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_SYMREF:\n-\t\t\tin.ref.value.symref = xstrdup(\"target\");\n+\t\t\tin.u.ref.value.symref = xstrdup(\"target\");\n \t\t\tbreak;\n \t\t}\n-\t\tin.ref.refname = xstrdup(\"refs/heads/master\");\n+\t\tin.u.ref.refname = xstrdup(\"refs/heads/master\");\n \n \t\ttest_copy(&in);\n \n@@ -152,7 +152,7 @@ static void test_reftable_ref_record_roundtrip(void)\n \t\tm = reftable_record_decode(&out, key, i, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_ref_record_equal(&in.ref, &out.ref,\n+\t\tEXPECT(reftable_ref_record_equal(&in.u.ref, &out.u.ref,\n \t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\treftable_record_release(&in);\n \n@@ -222,23 +222,23 @@ static void test_reftable_log_record_roundtrip(void)\n \t\t/* populate out, to check for leaks. */\n \t\tstruct reftable_record out = {\n \t\t\t.type = BLOCK_TYPE_LOG,\n-\t\t\t.log = {\n+\t\t\t.u.log = {\n \t\t\t\t.refname = xstrdup(\"old name\"),\n-\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n-\t\t\t.value = {\n-\t\t\t\t.update = {\n-\t\t\t\t\t.new_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n-\t\t\t\t\t.old_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n-\t\t\t\t\t.name = xstrdup(\"old name\"),\n-\t\t\t\t\t.email = xstrdup(\"old@email\"),\n-\t\t\t\t\t.message = xstrdup(\"old message\"),\n+\t\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n+\t\t\t\t.value = {\n+\t\t\t\t\t.update = {\n+\t\t\t\t\t\t.new_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t\t.old_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t\t.name = xstrdup(\"old name\"),\n+\t\t\t\t\t\t.email = xstrdup(\"old@email\"),\n+\t\t\t\t\t\t.message = xstrdup(\"old message\"),\n+\t\t\t\t\t},\n \t\t\t\t},\n \t\t\t},\n-\t\t\t},\n \t\t};\n \t\tint n, m, valtype;\n \n-\t\trec.log = in[i];\n+\t\trec.u.log = in[i];\n \n \t\ttest_copy(&rec);\n \n@@ -251,7 +251,7 @@ static void test_reftable_log_record_roundtrip(void)\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_log_record_equal(&in[i], &out.log,\n+\t\tEXPECT(reftable_log_record_equal(&in[i], &out.u.log,\n \t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\treftable_log_record_release(&in[i]);\n \t\tstrbuf_release(&key);\n@@ -330,7 +330,7 @@ static void test_reftable_obj_record_roundtrip(void)\n \t\t};\n \t\tstruct reftable_record in = {\n \t\t\t.type = BLOCK_TYPE_OBJ,\n-\t\t\t.obj = recs[i],\n+\t\t\t.u.obj = recs[i],\n \t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n \t\tstruct reftable_record out = {\n@@ -348,13 +348,13 @@ static void test_reftable_obj_record_roundtrip(void)\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(in.obj.hash_prefix_len == out.obj.hash_prefix_len);\n-\t\tEXPECT(in.obj.offset_len == out.obj.offset_len);\n+\t\tEXPECT(in.u.obj.hash_prefix_len == out.u.obj.hash_prefix_len);\n+\t\tEXPECT(in.u.obj.offset_len == out.u.obj.offset_len);\n \n-\t\tEXPECT(!memcmp(in.obj.hash_prefix, out.obj.hash_prefix,\n-\t\t\t in.obj.hash_prefix_len));\n-\t\tEXPECT(0 == memcmp(in.obj.offsets, out.obj.offsets,\n-\t\t\t\t sizeof(uint64_t) * in.obj.offset_len));\n+\t\tEXPECT(!memcmp(in.u.obj.hash_prefix, out.u.obj.hash_prefix,\n+\t\t\t in.u.obj.hash_prefix_len));\n+\t\tEXPECT(0 == memcmp(in.u.obj.offsets, out.u.obj.offsets,\n+\t\t\t\t sizeof(uint64_t) * in.u.obj.offset_len));\n \t\tstrbuf_release(&key);\n \t\treftable_record_release(&out);\n \t}\n@@ -364,7 +364,7 @@ static void test_reftable_index_record_roundtrip(void)\n {\n \tstruct reftable_record in = {\n \t\t.type = BLOCK_TYPE_INDEX,\n-\t\t.idx = {\n+\t\t.u.idx = {\n \t\t\t.offset = 42,\n \t\t\t.last_key = STRBUF_INIT,\n \t\t},\n@@ -377,28 +377,28 @@ static void test_reftable_index_record_roundtrip(void)\n \tstruct strbuf key = STRBUF_INIT;\n \tstruct reftable_record out = {\n \t\t.type = BLOCK_TYPE_INDEX,\n-\t\t.idx = { .last_key = STRBUF_INIT },\n+\t\t.u.idx = { .last_key = STRBUF_INIT },\n \t};\n \tint n, m;\n \tuint8_t extra;\n \n-\tstrbuf_addstr(&in.idx.last_key, \"refs/heads/master\");\n+\tstrbuf_addstr(&in.u.idx.last_key, \"refs/heads/master\");\n \treftable_record_key(&in, &key);\n \ttest_copy(&in);\n \n-\tEXPECT(0 == strbuf_cmp(&key, &in.idx.last_key));\n+\tEXPECT(0 == strbuf_cmp(&key, &in.u.idx.last_key));\n \tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(n > 0);\n \n \textra = reftable_record_val_type(&in);\n \tm = reftable_record_decode(&out, key, extra, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(m == n);\n \n-\tEXPECT(in.idx.offset == out.idx.offset);\n+\tEXPECT(in.u.idx.offset == out.u.idx.offset);\n \n \treftable_record_release(&out);\n \tstrbuf_release(&key);\n-\tstrbuf_release(&in.idx.last_key);\n+\tstrbuf_release(&in.u.idx.last_key);\n }\n \n int record_test_main(int argc, const char *argv[])\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex df256f3a8b..f9544c5f97 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -253,7 +253,7 @@ int reftable_writer_add_ref(struct reftable_writer *w,\n {\n \tstruct reftable_record rec = {\n \t\t.type = BLOCK_TYPE_REF,\n-\t\t.ref = *ref,\n+\t\t.u.ref = *ref,\n \t};\n \tint err = 0;\n \n@@ -263,7 +263,7 @@ int reftable_writer_add_ref(struct reftable_writer *w,\n \t ref->update_index > w->max_update_index)\n \t\treturn REFTABLE_API_ERROR;\n \n-\trec.ref.update_index -= w->min_update_index;\n+\trec.u.ref.update_index -= w->min_update_index;\n \n \terr = writer_add_record(w, &rec);\n \tif (err < 0)\n@@ -304,7 +304,7 @@ static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n {\n \tstruct reftable_record rec = {\n \t\t.type = BLOCK_TYPE_LOG,\n-\t\t.log = *log,\n+\t\t.u.log = *log,\n \t};\n \tif (w->block_writer &&\n \t block_writer_type(w->block_writer) == BLOCK_TYPE_REF) {\n@@ -397,7 +397,7 @@ static int writer_finish_section(struct reftable_writer *w)\n \t\tfor (i = 0; i < idx_len; i++) {\n \t\t\tstruct reftable_record rec = {\n \t\t\t\t.type = BLOCK_TYPE_INDEX,\n-\t\t\t\t.idx = idx[i],\n+\t\t\t\t.u.idx = idx[i],\n \t\t\t};\n \t\t\tif (block_writer_add(w->block_writer, &rec) == 0) {\n \t\t\t\tcontinue;\n@@ -468,7 +468,7 @@ static void write_object_record(void *void_arg, void *key)\n \tstruct obj_index_tree_node *entry = key;\n \tstruct reftable_record rec = {\n \t\t.type = BLOCK_TYPE_OBJ,\n-\t\t.obj = {\n+\t\t.u.obj = {\n \t\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n \t\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n \t\t\t.offsets = entry->offsets,\n@@ -491,7 +491,7 @@ static void write_object_record(void *void_arg, void *key)\n \tif (arg->err == 0)\n \t\tgoto done;\n \n-\trec.obj.offset_len = 0;\n+\trec.u.obj.offset_len = 0;\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \n \t/* Should be able to write into a fresh block. */\n"},{"id":"443402","messageId":"xmqq4k7j68eg.fsf@gitster.g","threadId":"57047","inReplyTo":"YbAVOtYXA1Hf9EtJ@coredump.intra.peff.net","subject":"Re: [PATCH 10/10] reftable: make reftable_record a tagged union","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-08T04:13:43Z","receivedAt":"2021-12-08T04:13:48Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n>> error: ISO C99 doesn't support unnamed structs/unions [-Werror=pedantic]\n>\n> Hmm. It's interesting that the regular DEVELOPER=1 doesn't catch this.\n> It's because we don't specify -std there, and newer gcc defaults to\n> gnu17 (unnamed unions appeared in c11, I think). I wonder if it would be\n> helpful to teach config.mak.dev to pass -std=c99.\n\nFWIW, I use -std=gnu99 as our Makefile suggests.\n"},{"id":"443414","messageId":"CAFQ2z_NuOy+-pfSoNAYjJhS9jZCYOfoFue10=k=iyPVsPYrB3g@mail.gmail.com","threadId":"57047","inReplyTo":"xmqq4k7j68eg.fsf@gitster.g","subject":"Re: [PATCH 10/10] reftable: make reftable_record a tagged union","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-12-08T10:30:11Z","receivedAt":"2021-12-08T10:30:25Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Wed, Dec 8, 2021 at 5:13 AM Junio C Hamano wrote:\n> >> error: ISO C99 doesn't support unnamed structs/unions [-Werror=pedantic]\n> >\n> > Hmm. It's interesting that the regular DEVELOPER=1 doesn't catch this.\n> > It's because we don't specify -std there, and newer gcc defaults to\n> > gnu17 (unnamed unions appeared in c11, I think). I wonder if it would be\n> > helpful to teach config.mak.dev to pass -std=c99.\n>\n> FWIW, I use -std=gnu99 as our Makefile suggests.\n\nI understand that the default build should be lenient rather than\nstrict for portability reasons. However, it would be good if the CI\nwas strict with this.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"443447","messageId":"a7b4c6aa-c8cd-eaa7-f4b5-dee0028cc4d2@gmail.com","threadId":"57047","inReplyTo":"603bd1d4f6e4ceda02a5165eedd3ae5b92b8929e.1638899124.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 02/10] reftable: fix resource leak in error path","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2021-12-08T14:30:38Z","receivedAt":"2021-12-08T14:30:47Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/7/2021 12:45 PM, Han-Wen Nienhuys via GitGitGadget wrote:\n> From: Han-Wen Nienhuys \n> \n> This would be triggered by corrupt files, so it doesn't have test coverage. This\n> was discovered by a Coverity scan.\n> \n> Signed-off-by: Han-Wen Nienhuys \n> ---\n> reftable/block.c | 25 ++++++++++++++++++-------\n> 1 file changed, 18 insertions(+), 7 deletions(-)\n> \n> diff --git a/reftable/block.c b/reftable/block.c\n> index 855e3f5c947..d7347bb3152 100644\n> --- a/reftable/block.c\n> +++ b/reftable/block.c\n> @@ -188,13 +188,16 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n> \tuint32_t full_block_size = table_block_size;\n> \tuint8_t typ = block->data[header_off];\n> \tuint32_t sz = get_be24(block->data + header_off + 1);\n> -\n> +\tint err = 0;\n> \tuint16_t restart_count = 0;\n> \tuint32_t restart_start = 0;\n> \tuint8_t *restart_bytes = NULL;\n> +\tuint8_t *uncompressed = NULL;\n\nYou define this here...\n\n> \n> -\tif (!reftable_is_block_type(typ))\n> -\t\treturn REFTABLE_FORMAT_ERROR;\n> +\tif (!reftable_is_block_type(typ)) {\n> +\t\terr = REFTABLE_FORMAT_ERROR;\n> +\t\tgoto done;\n> +\t}\n> \n> \tif (typ == BLOCK_TYPE_LOG) {\n> \t\tint block_header_skip = 4 + header_off;\n> @@ -213,15 +216,19 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n> \t\t uncompress2(uncompressed + block_header_skip, &dst_len,\n> \t\t\t\tblock->data + block_header_skip, &src_len)) {\n> \t\t\treftable_free(uncompressed);\n\nBut it is already used here, because it is defined within the if()\nblock. You need to remove that definition, too, or your change here\ndoes nothing.\n\n...\n\n> \t\t/* We're done with the input data. */\n> \t\treftable_block_done(block);\n> \t\tblock->data = uncompressed;\n> +\t\tuncompressed = NULL;\n\nFor example, this nullifies the local version of uncompressed.\n\n(I had to double-check that this even compiled, which was\nsurprising to me.)\n\n> +done:\n> +\tif (uncompressed) {\n> +\t\treftable_free(uncompressed);\n> +\t}\n> +\treturn err;\n\nRemove that local declaration and this will be a good patch.\n\nThanks,\n-Stolee\n"},{"id":"443448","messageId":"e47990d1-4efd-e0ae-5696-bec9a854ea1d@gmail.com","threadId":"57047","inReplyTo":"458adf0fa9b90dae831da7aa8cdb67fa9a064ab5.1638899124.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 08/10] reftable: order unittests by complexity","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2021-12-08T14:32:56Z","receivedAt":"2021-12-08T14:32:59Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/7/2021 12:45 PM, Han-Wen Nienhuys via GitGitGadget wrote:\n> From: Han-Wen Nienhuys \n...\n> int cmd__reftable(int argc, const char **argv)\n> {\n> +\t// test from simple to complex.\n\nUse /* ... */ style comments, even for a single line.\n\nThanks,\n-Stolee\n"},{"id":"443449","messageId":"072ff09c-9174-e769-7ebb-4bb248199337@gmail.com","threadId":"57047","inReplyTo":"8deccc3a1dff7e4f7d613fa63d2781fd1f11f841.1638899124.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 10/10] reftable: make reftable_record a tagged union","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2021-12-08T14:35:36Z","receivedAt":"2021-12-08T14:35:39Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/7/2021 12:45 PM, Han-Wen Nienhuys via GitGitGadget wrote:\n> From: Han-Wen Nienhuys \n> \n> This reduces the amount of glue code, because we don't need a void pointer or\n> vtable within the structure.\n> \n> The only snag is that reftable_index_record contain a strbuf, so it cannot be\n> zero-initialized. To address this, introduce reftable_record_for() to create a\n> fresh instance, given a record type.\n> \n> Thanks to Peff for the suggestion.\n> \n> Signed-off-by: Han-Wen Nienhuys \n> ---\n> reftable/block.c | 4 +-\n> reftable/block_test.c | 22 +++---\n> reftable/generic.c | 35 ++++----\n> reftable/iter.c | 4 +-\n> reftable/merged.c | 37 ++++-----\n> reftable/pq.c | 3 +-\n> reftable/pq_test.c | 31 ++++----\n> reftable/reader.c | 105 ++++++++++++------------\n> reftable/record.c | 176 ++++++++++++++++-------------------------\n> reftable/record.h | 45 +++++------\n> reftable/record_test.c | 162 +++++++++++++++++++------------------\n> reftable/writer.c | 46 ++++++-----\n\nThis is a HUGE diff, especially compared to the previous changes\nin this series. I recommend splitting this out into its own series\nand finding a way to break it down into smaller changes.\n\nThanks,\n-Stolee\n"},{"id":"443450","messageId":"CAFQ2z_PmwLKC2Y39yj2Cqk=FMGC4gJTgZUEFOmgC=wE2KmdTfQ@mail.gmail.com","threadId":"57047","inReplyTo":"072ff09c-9174-e769-7ebb-4bb248199337@gmail.com","subject":"Re: [PATCH 10/10] reftable: make reftable_record a tagged union","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-12-08T14:48:08Z","receivedAt":"2021-12-08T14:48:22Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Wed, Dec 8, 2021 at 3:35 PM Derrick Stolee wrote:\n> This is a HUGE diff, especially compared to the previous changes\n> in this series. I recommend splitting this out into its own series\n> and finding a way to break it down into smaller changes.\n\nWould you have a suggestion how? The reftable_record type is used\nacross the reftable library, so if we change its definition, that\nimpacts most callsites.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"443479","messageId":"xmqq1r2n3vhj.fsf@gitster.g","threadId":"57047","inReplyTo":"CAFQ2z_NuOy+-pfSoNAYjJhS9jZCYOfoFue10=k=iyPVsPYrB3g@mail.gmail.com","subject":"Re: [PATCH 10/10] reftable: make reftable_record a tagged union","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-08T16:35:36Z","receivedAt":"2021-12-08T16:35:52Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Han-Wen Nienhuys writes:\n\n> On Wed, Dec 8, 2021 at 5:13 AM Junio C Hamano wrote:\n>> >> error: ISO C99 doesn't support unnamed structs/unions [-Werror=pedantic]\n>> >\n>> > Hmm. It's interesting that the regular DEVELOPER=1 doesn't catch this.\n>> > It's because we don't specify -std there, and newer gcc defaults to\n>> > gnu17 (unnamed unions appeared in c11, I think). I wonder if it would be\n>> > helpful to teach config.mak.dev to pass -std=c99.\n>>\n>> FWIW, I use -std=gnu99 as our Makefile suggests.\n>\n> I understand that the default build should be lenient rather than\n> strict for portability reasons. However, it would be good if the CI\n> was strict with this.\n\nYeah, I agree with the above, and was a bit surprised that I found\nthe issue via my local build after it came from GGG.\n\n\n"},{"id":"443481","messageId":"xmqqv8zz2ge1.fsf@gitster.g","threadId":"57047","inReplyTo":"072ff09c-9174-e769-7ebb-4bb248199337@gmail.com","subject":"Re: [PATCH 10/10] reftable: make reftable_record a tagged union","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-08T16:47:02Z","receivedAt":"2021-12-08T16:47:09Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Derrick Stolee writes:\n\n> On 12/7/2021 12:45 PM, Han-Wen Nienhuys via GitGitGadget wrote:\n>> From: Han-Wen Nienhuys \n>> \n>> This reduces the amount of glue code, because we don't need a void pointer or\n>> vtable within the structure.\n>> \n>> The only snag is that reftable_index_record contain a strbuf, so it cannot be\n>> zero-initialized. To address this, introduce reftable_record_for() to create a\n>> fresh instance, given a record type.\n>> \n>> Thanks to Peff for the suggestion.\n>> \n>> Signed-off-by: Han-Wen Nienhuys \n>> ---\n>> reftable/block.c | 4 +-\n>> reftable/block_test.c | 22 +++---\n>> reftable/generic.c | 35 ++++----\n>> reftable/iter.c | 4 +-\n>> reftable/merged.c | 37 ++++-----\n>> reftable/pq.c | 3 +-\n>> reftable/pq_test.c | 31 ++++----\n>> reftable/reader.c | 105 ++++++++++++------------\n>> reftable/record.c | 176 ++++++++++++++++-------------------------\n>> reftable/record.h | 45 +++++------\n>> reftable/record_test.c | 162 +++++++++++++++++++------------------\n>> reftable/writer.c | 46 ++++++-----\n>\n> This is a HUGE diff, especially compared to the previous changes\n> in this series. I recommend splitting this out into its own series\n> and finding a way to break it down into smaller changes.\n\nAs the reftable_record structure is used everywhere (and that is why\nthis step has to touch everywhere), I suspect that a reviewable fix\nin small chunks would be achievable only if we redo the topic that\nintroduces this hierarchy and fix the type at the source, as if the\nreftable_record structure was a struct with union in it from the\nbeginning, I am afraid.\n\nPerhaps reftable_record_for() can be implemented without changing\nthe shape of the underlying reftable_record structure in an earlier\nstep, then all the users of reftable_record instances can be\nmigrated to call it, and then finally the shape of the structure and\nthe implementation of reftable_record_for() can be updated? \n\nIf that is doable, then the \"migrate each users\" part can be split\npurely by size. But (1) I do not know if the first step is even\ndoable, and (2) I am not sure if it is worth going a somewhat\nroundabout route to get to the same destination in this case.\n\nSo...\n\n\n"},{"id":"443488","messageId":"CAFQ2z_OMK=w8j9a3BzQJxGQdc0RXY-zUoaBC08C-mjWiD+Vp1g@mail.gmail.com","threadId":"57047","inReplyTo":"xmqqv8zz2ge1.fsf@gitster.g","subject":"Re: [PATCH 10/10] reftable: make reftable_record a tagged union","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-12-08T17:51:55Z","receivedAt":"2021-12-08T17:52:12Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Wed, Dec 8, 2021 at 5:47 PM Junio C Hamano wrote:\n> Perhaps reftable_record_for() can be implemented without changing\n> the shape of the underlying reftable_record structure in an earlier\n> step, then all the users of reftable_record instances can be\n> migrated to call it, and then finally the shape of the structure and\n> the implementation of reftable_record_for() can be updated?\n\nWe are going from\n\n reftable_ref_record ref = { 0 };\n reftable_record rec = { 0 };\n record_from_ref_record(&rec, &ref);\n\nto\n\n reftable_record rec = { .type = BLOCK_TYPE_REF };\n\nwhich is why this change is so nice. reftable_record_for() is used in\njust a few places, so it wouldn't make the change much smaller.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\nRegistergericht und -nummer: Hamburg, HRB 86891\nSitz der Gesellschaft: Hamburg\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"443491","messageId":"569fded7-48d7-6f48-47fb-b78c403214fc@gmail.com","threadId":"57047","inReplyTo":"CAFQ2z_PmwLKC2Y39yj2Cqk=FMGC4gJTgZUEFOmgC=wE2KmdTfQ@mail.gmail.com","subject":"Re: [PATCH 10/10] reftable: make reftable_record a tagged union","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2021-12-08T18:17:41Z","receivedAt":"2021-12-08T18:17:46Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/8/2021 9:48 AM, Han-Wen Nienhuys wrote:\n> On Wed, Dec 8, 2021 at 3:35 PM Derrick Stolee wrote:\n>> This is a HUGE diff, especially compared to the previous changes\n>> in this series. I recommend splitting this out into its own series\n>> and finding a way to break it down into smaller changes.\n> \n> Would you have a suggestion how? The reftable_record type is used\n> across the reftable library, so if we change its definition, that\n> impacts most callsites.\n \nLooking at the diff, I'm seeing a lot of things that don't seem\nnecessary to changing the struct, like renaming \"reftable_new_record\"\nto \"reftable_record_for\". Yes, the implementation changes, but that\nimplementation is related to the struct changing and can be done in\na more isolated way.\n\nSome things can be split out easier, such as the addition of\n\"uint8_t type\" into the struct and replacing all reftable_record_type()\ncalls with .type.\n\nThe big one is definitely the addition of a union, but it could be\ndone in a way that only changes the implementation of methods with\nnames such as reftable_record_from_obj() then we can replace those with\n\".obj\" later.\n\nSome of these changes are definitely going to have a lot of lines,\nbut when it's the same kind of change it is easy to verify. I'm\ngetting lost in this diff because it's doing too many things at once.\n\nThanks,\n-Stolee\n"},{"id":"443510","messageId":"YbEMnksMEuAz3Nt0@coredump.intra.peff.net","threadId":"57047","inReplyTo":"xmqq4k7j68eg.fsf@gitster.g","subject":"[PATCH] config.mak.dev: specify -std=gnu99 for gcc/clang","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-12-08T19:50:54Z","receivedAt":"2021-12-08T19:50:56Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Dec 07, 2021 at 08:13:43PM -0800, Junio C Hamano wrote:\n\n> Jeff King writes:\n> \n> >> error: ISO C99 doesn't support unnamed structs/unions [-Werror=pedantic]\n> >\n> > Hmm. It's interesting that the regular DEVELOPER=1 doesn't catch this.\n> > It's because we don't specify -std there, and newer gcc defaults to\n> > gnu17 (unnamed unions appeared in c11, I think). I wonder if it would be\n> > helpful to teach config.mak.dev to pass -std=c99.\n> \n> FWIW, I use -std=gnu99 as our Makefile suggests.\n\nI think the patch below would have detected this both locally for\nHan-Wen, but also in C.\n\n-- >8 --\nSubject: [PATCH] config.mak.dev: specify -std=gnu99 for gcc/clang\n\nThe point of DEVELOPER=1 is to turn up the warnings so we can catch\nportability or correctness mistakes at the compiler level. But since\nmodern compilers tend to default to modern standards like gnu17, we\nmight miss warnings about older standards, even though we expect Git to\nbuild with compilers that use them.\n\nSo it's helpful for developer builds to set the -std argument to our\nlowest-common denominator. Traditionally this was c89, but since we're\nmoving to assuming c99 in 7bc341e21b (git-compat-util: add a test\nballoon for C99 support, 2021-12-01) that seems like a good spot to\nland. And as explained in that commit, we want \"gnu99\" because we still\nwant to take advantage of some extensions when they're available.\n\nThe new argument kicks in only for clang and gcc (which we know to\nsupport \"-std=\" and \"gnu\" standards). And only for compiler versions\nwhich default to a newer standard. That will avoid accidentally\nsilencing any build problems that non-developers would run into on older\ncompilers that default to c89.\n\nMy digging found that the default switched to gnu11 in gcc 5.1.0.\nClang's documentation is less clear, but has done so since at least\nclang-7. So that's what I put in the conditional here. It's OK to err on\nthe side of not-enabling this for older compilers. Most developers (as\nwell as CI) are using much more recent versions, so any warnings will\neventually surface.\n\nA concrete example is anonymous unions, which became legal in c11.\nWithout this patch, \"gcc -pedantic\" will not complain about them, but\nwill if we add in \"-std=gnu99\".\n\nSigned-off-by: Jeff King \n---\n config.mak.dev | 5 +++++\n 1 file changed, 5 insertions(+)\n\ndiff --git a/config.mak.dev b/config.mak.dev\nindex 7673fed114..d4afac6b51 100644\n--- a/config.mak.dev\n+++ b/config.mak.dev\n@@ -19,6 +19,11 @@ endif\n endif\n endif\n endif\n+\n+ifneq ($(or $(filter gcc6,$(COMPILER_FEATURES)),$(filter clang7,$(COMPILER_FEATURES))),)\n+DEVELOPER_CFLAGS += -std=gnu99\n+endif\n+\n DEVELOPER_CFLAGS += -Wdeclaration-after-statement\n DEVELOPER_CFLAGS += -Wformat-security\n DEVELOPER_CFLAGS += -Wold-style-definition\n-- \n2.34.1.503.gbf394de69d\n\n"},{"id":"443521","messageId":"7c033815183cc8ad7a76bc2b9456b17041107c2c.1639000187.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v2.git.git.1639000187.gitgitgadget@gmail.com","subject":"[PATCH v2 01/11] reftable: fix OOB stack write in print functions","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-08T21:49:36Z","receivedAt":"2021-12-08T21:49:53Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 6a5dac32dc6..8536bd03aa9 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -254,7 +254,7 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)\n void reftable_ref_record_print(struct reftable_ref_record *ref,\n \t\t\t uint32_t hash_id)\n {\n-\tchar hex[2 * GIT_SHA256_RAWSZ + 1] = { 0 }; /* BUG */\n+\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */\n \tprintf(\"ref{%s(%\" PRIu64 \") \", ref->refname, ref->update_index);\n \tswitch (ref->value_type) {\n \tcase REFTABLE_REF_SYMREF:\n@@ -586,7 +586,7 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {\n void reftable_log_record_print(struct reftable_log_record *log,\n \t\t\t uint32_t hash_id)\n {\n-\tchar hex[GIT_SHA256_RAWSZ + 1] = { 0 };\n+\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 };\n \n \tswitch (log->value_type) {\n \tcase REFTABLE_LOG_DELETION:\n-- \ngitgitgadget\n\n"},{"id":"443523","messageId":"pull.1152.v2.git.git.1639000187.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.git.git.1638899124.gitgitgadget@gmail.com","subject":"[PATCH v2 00/11] Reftable coverity fixes","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-08T21:49:35Z","receivedAt":"2021-12-08T21:49:53Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"This series was targeted to 'next'.\n\nThis addresses some complaints from Coverity that Peff reported.\n\nCI on GGG shows a segfault on windows\n(https://github.com/git/git/runs/4446635428?check_suite_focus=true), which I\ncan't make sense of. Any hints would be greatly appreciated.\n\nHan-Wen Nienhuys (11):\n reftable: fix OOB stack write in print functions\n reftable: fix resource leak in error path\n reftable: fix resource leak blocksource.c\n reftable: check reftable_stack_auto_compact() return value\n reftable: ignore remove() return value in stack_test.c\n reftable: fix resource warning\n reftable: fix NULL derefs in error paths\n reftable: order unittests by complexity\n reftable: drop stray printf in readwrite_test\n reftable: handle null refnames in reftable_ref_record_equal\n reftable: make reftable_record a tagged union\n\n reftable/block.c | 31 +++++--\n reftable/block_test.c | 22 ++---\n reftable/blocksource.c | 6 +-\n reftable/generic.c | 35 +++----\n reftable/iter.c | 4 +-\n reftable/merged.c | 37 ++++----\n reftable/pq.c | 3 +-\n reftable/pq_test.c | 31 ++++---\n reftable/reader.c | 108 +++++++++++-----------\n reftable/readwrite_test.c | 1 -\n reftable/record.c | 188 +++++++++++++++-----------------------\n reftable/record.h | 45 ++++-----\n reftable/record_test.c | 181 ++++++++++++++++++------------------\n reftable/stack.c | 15 +--\n reftable/stack_test.c | 3 +-\n reftable/writer.c | 46 ++++++----\n t/helper/test-reftable.c | 9 +-\n 17 files changed, 376 insertions(+), 389 deletions(-)\n\n\nbase-commit: a8338297339b1ab064e799c15c4fc56c122ef967\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1152%2Fhanwen%2Freftable-coverity-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1152/hanwen/reftable-coverity-v2\nPull-Request: https://github.com/git/git/pull/1152\n\nRange-diff vs v1:\n\n 1: 7c033815183 = 1: 7c033815183 reftable: fix OOB stack write in print functions\n 2: 603bd1d4f6e ! 2: 1ddcfe61ebc reftable: fix resource leak in error path\n @@ reftable/block.c: int block_reader_init(struct block_reader *br, struct reftable\n \n \tif (typ == BLOCK_TYPE_LOG) {\n \t\tint block_header_skip = 4 + header_off;\n +@@ reftable/block.c: int block_reader_init(struct block_reader *br, struct reftable_block *block,\n + \t\tuLongf src_len = block->len - block_header_skip;\n + \t\t/* Log blocks specify the *uncompressed* size in their header.\n + \t\t */\n +-\t\tuint8_t *uncompressed = reftable_malloc(sz);\n ++\t\tuncompressed = reftable_malloc(sz);\n + \n + \t\t/* Copy over the block header verbatim. It's not compressed. */\n + \t\tmemcpy(uncompressed, block->data, block_header_skip);\n @@ reftable/block.c: int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\t uncompress2(uncompressed + block_header_skip, &dst_len,\n \t\t\t\tblock->data + block_header_skip, &src_len)) {\n 3: 97279040dde = 3: e052b2a61d6 reftable: fix resource leak blocksource.c\n 4: 9288289ebcb = 4: 9063137457b reftable: check reftable_stack_auto_compact() return value\n 5: 9269977a1c3 = 5: 5020be156ae reftable: ignore remove() return value in stack_test.c\n 6: eab71ab008e = 6: 64c18d01cad reftable: fix resource warning\n 7: 47fef87eb4a = 7: 700387ac5d3 reftable: fix NULL derefs in error paths\n 8: 458adf0fa9b = 8: 713f1d09f68 reftable: order unittests by complexity\n 9: cd5d8d3607d = 9: cb601b51a47 reftable: drop stray printf in readwrite_test\n -: ----------- > 10: a0f83eff19f reftable: handle null refnames in reftable_ref_record_equal\n 10: 8deccc3a1df ! 11: a2743033cfd reftable: make reftable_record a tagged union\n @@ Commit message\n \n Thanks to Peff for the suggestion.\n \n + Helped-by: Jeff King \n Signed-off-by: Han-Wen Nienhuys \n \n ## reftable/block.c ##\n @@ reftable/block_test.c: static void test_block_read_write(void)\n -\t\tref.refname = name;\n -\t\tref.value_type = REFTABLE_REF_VAL1;\n -\t\tref.value.val1 = hash;\n -+\t\trec.ref.refname = name;\n -+\t\trec.ref.value_type = REFTABLE_REF_VAL1;\n -+\t\trec.ref.value.val1 = hash;\n ++\t\trec.u.ref.refname = name;\n ++\t\trec.u.ref.value_type = REFTABLE_REF_VAL1;\n ++\t\trec.u.ref.value.val1 = hash;\n \n \t\tnames[i] = xstrdup(name);\n \t\tn = block_writer_add(&bw, &rec);\n -\t\tref.refname = NULL;\n -\t\tref.value_type = REFTABLE_REF_DELETION;\n -+\t\trec.ref.refname = NULL;\n -+\t\trec.ref.value_type = REFTABLE_REF_DELETION;\n ++\t\trec.u.ref.refname = NULL;\n ++\t\trec.u.ref.value_type = REFTABLE_REF_DELETION;\n \t\tEXPECT(n == 0);\n \t}\n \n @@ reftable/block_test.c: static void test_block_read_write(void)\n \t\t\tbreak;\n \t\t}\n -\t\tEXPECT_STREQ(names[j], ref.refname);\n -+\t\tEXPECT_STREQ(names[j], rec.ref.refname);\n ++\t\tEXPECT_STREQ(names[j], rec.u.ref.refname);\n \t\tj++;\n \t}\n \n @@ reftable/block_test.c: static void test_block_read_write(void)\n \t\tEXPECT(n == 0);\n \n -\t\tEXPECT_STREQ(names[i], ref.refname);\n -+\t\tEXPECT_STREQ(names[i], rec.ref.refname);\n ++\t\tEXPECT_STREQ(names[i], rec.u.ref.refname);\n \n \t\twant.len--;\n \t\tn = block_reader_seek(&br, &it, &want);\n @@ reftable/block_test.c: static void test_block_read_write(void)\n \t\tn = block_iter_next(&it, &rec);\n \t\tEXPECT(n == 0);\n -\t\tEXPECT_STREQ(names[10 * (i / 10)], ref.refname);\n -+\t\tEXPECT_STREQ(names[10 * (i / 10)], rec.ref.refname);\n ++\t\tEXPECT_STREQ(names[10 * (i / 10)], rec.u.ref.refname);\n \n \t\tblock_iter_close(&it);\n \t}\n @@ reftable/generic.c: https://developers.google.com/open-source/licenses/bsd\n -\tstruct reftable_record rec = { NULL };\n -\treftable_record_from_ref(&rec, &ref);\n +\tstruct reftable_record rec = { .type = BLOCK_TYPE_REF,\n -+\t\t\t\t .ref = {\n ++\t\t\t\t .u.ref = {\n +\t\t\t\t\t .refname = (char *)name,\n +\t\t\t\t } };\n \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n @@ reftable/generic.c: https://developers.google.com/open-source/licenses/bsd\n -\tstruct reftable_record rec = { NULL };\n -\treftable_record_from_log(&rec, &log);\n +\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n -+\t\t\t\t .log = {\n ++\t\t\t\t .u.log = {\n +\t\t\t\t\t .refname = (char *)name,\n +\t\t\t\t\t .update_index = ~((uint64_t)0),\n +\t\t\t\t } };\n @@ reftable/generic.c: void reftable_iterator_destroy(struct reftable_iterator *it)\n -\treturn iterator_next(it, &rec);\n +\tstruct reftable_record rec = { .type = BLOCK_TYPE_REF };\n +\tint err = iterator_next(it, &rec);\n -+\t*ref = rec.ref;\n ++\t*ref = rec.u.ref;\n +\treturn err;\n }\n \n @@ reftable/generic.c: void reftable_iterator_destroy(struct reftable_iterator *it)\n -\treturn iterator_next(it, &rec);\n +\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG };\n +\tint err = iterator_next(it, &rec);\n -+\t*log = rec.log;\n ++\t*log = rec.u.log;\n +\treturn err;\n }\n \n @@ reftable/iter.c: static int filtering_ref_iterator_next(void *iter_arg,\n {\n \tstruct filtering_ref_iterator *fri = iter_arg;\n -\tstruct reftable_ref_record *ref = rec->data;\n -+\tstruct reftable_ref_record *ref = &rec->ref;\n ++\tstruct reftable_ref_record *ref = &rec->u.ref;\n \tint err = 0;\n \twhile (1) {\n \t\terr = reftable_iterator_next_ref(&fri->it, ref);\n @@ reftable/iter.c: static int indexed_table_ref_iter_next_block(struct indexed_tab\n {\n \tstruct indexed_table_ref_iter *it = p;\n -\tstruct reftable_ref_record *ref = rec->data;\n -+\tstruct reftable_ref_record *ref = &rec->ref;\n ++\tstruct reftable_ref_record *ref = &rec->u.ref;\n \n \twhile (1) {\n \t\tint err = block_iter_next(&it->cur, rec);\n @@ reftable/merged.c: int reftable_merged_table_seek_ref(struct reftable_merged_tab\n -\t\t.refname = (char *)name,\n +\tstruct reftable_record rec = {\n +\t\t.type = BLOCK_TYPE_REF,\n -+\t\t.ref = {\n ++\t\t.u.ref = {\n +\t\t\t.refname = (char *)name,\n +\t\t},\n \t};\n @@ reftable/merged.c: int reftable_merged_table_seek_log_at(struct reftable_merged_\n -\t\t.update_index = update_index,\n +\tstruct reftable_record rec = {\n +\t\t.type = BLOCK_TYPE_LOG,\n -+\t\t.log = {\n ++\t\t.u.log = {\n +\t\t\t.refname = (char *)name,\n +\t\t\t.update_index = update_index,\n +\t\t}\n @@ reftable/pq_test.c: static void test_pq(void)\n +\t\tstruct pq_entry e = {\n +\t\t\t.rec = {\n +\t\t\t\t.type = BLOCK_TYPE_REF,\n -+\t\t\t\t.ref = {\n ++\t\t\t\t.u.ref = {\n +\t\t\t\t\t.refname = names[i],\n +\t\t\t\t}\n +\t\t\t}\n @@ reftable/pq_test.c: static void test_pq(void)\n +\t\tEXPECT(rec->type == BLOCK_TYPE_REF);\n \t\tif (last) {\n -\t\t\tEXPECT(strcmp(last, ref->refname) < 0);\n -+\t\t\tEXPECT(strcmp(last, rec->ref.refname) < 0);\n ++\t\t\tEXPECT(strcmp(last, rec->u.ref.refname) < 0);\n \t\t}\n -\t\tlast = ref->refname;\n -\t\tref->refname = NULL;\n -\t\treftable_free(ref);\n +\t\t// this is names[i], so don't dealloc.\n -+\t\tlast = rec->ref.refname;\n -+\t\trec->ref.refname = NULL;\n ++\t\tlast = rec->u.ref.refname;\n ++\t\trec->u.ref.refname = NULL;\n +\t\treftable_record_release(rec);\n \t}\n -\n @@ reftable/reader.c: static int table_iter_next_in_block(struct table_iter *ti,\n -\t\t((struct reftable_ref_record *)rec->data)->update_index +=\n -\t\t\tti->r->min_update_index;\n +\tif (res == 0 && rec->type == BLOCK_TYPE_REF) {\n -+\t\trec->ref.update_index += ti->r->min_update_index;\n ++\t\trec->u.ref.update_index += ti->r->min_update_index;\n \t}\n \n \treturn res;\n @@ reftable/reader.c: static int reader_seek_indexed(struct reftable_reader *r,\n -\tstruct reftable_record index_result_rec = { NULL };\n +\tstruct reftable_record want_index = {\n +\t\t.type = BLOCK_TYPE_INDEX,\n -+\t\t.idx = { .last_key = STRBUF_INIT }\n ++\t\t.u.idx = { .last_key = STRBUF_INIT }\n +\t};\n +\tstruct reftable_record index_result = {\n +\t\t.type = BLOCK_TYPE_INDEX,\n -+\t\t.idx = { .last_key = STRBUF_INIT },\n ++\t\t.u.idx = { .last_key = STRBUF_INIT },\n +\t};\n \tstruct table_iter index_iter = TABLE_ITER_INIT;\n \tstruct table_iter next = TABLE_ITER_INIT;\n @@ reftable/reader.c: static int reader_seek_indexed(struct reftable_reader *r,\n -\treftable_record_from_index(&index_result_rec, &index_result);\n -\n -\terr = reader_start(r, &index_iter, reftable_record_type(rec), 1);\n -+\treftable_record_key(rec, &want_index.idx.last_key);\n ++\treftable_record_key(rec, &want_index.u.idx.last_key);\n +\terr = reader_start(r, &index_iter, rec->type, 1);\n \tif (err < 0)\n \t\tgoto done;\n @@ reftable/reader.c: static int reader_seek_indexed(struct reftable_reader *r,\n \t\t\tgoto done;\n \n -\t\terr = reader_table_iter_at(r, &next, index_result.offset, 0);\n -+\t\terr = reader_table_iter_at(r, &next, index_result.idx.offset,\n ++\t\terr = reader_table_iter_at(r, &next, index_result.u.idx.offset,\n +\t\t\t\t\t 0);\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n -\t\terr = block_iter_seek(&next.bi, &want_index.last_key);\n -+\t\terr = block_iter_seek(&next.bi, &want_index.idx.last_key);\n ++\t\terr = block_iter_seek(&next.bi, &want_index.u.idx.last_key);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n @@ reftable/reader.c: static int reader_seek(struct reftable_reader *r, struct reft\n -\t\t.refname = (char *)name,\n +\tstruct reftable_record rec = {\n +\t\t.type = BLOCK_TYPE_REF,\n -+\t\t.ref = {\n ++\t\t.u.ref = {\n +\t\t\t.refname = (char *)name,\n +\t\t},\n \t};\n @@ reftable/reader.c: int reftable_reader_seek_log_at(struct reftable_reader *r,\n -\t\t.update_index = update_index,\n +\tstruct reftable_record rec = {\n +\t\t.type = BLOCK_TYPE_LOG,\n -+\t\t.log = {\n ++\t\t.u.log = {\n +\t\t\t.refname = (char *)name,\n +\t\t\t.update_index = update_index,\n +\t\t}\n @@ reftable/reader.c: static int reftable_reader_refs_for_indexed(struct reftable_r\n -\t\t.hash_prefix_len = r->object_id_len,\n +\tstruct reftable_record want = {\n +\t\t.type = BLOCK_TYPE_OBJ,\n -+\t\t.obj = {\n ++\t\t.u.obj = {\n +\t\t\t.hash_prefix = oid,\n +\t\t\t.hash_prefix_len = r->object_id_len,\n +\t\t},\n @@ reftable/reader.c: static int reftable_reader_refs_for_indexed(struct reftable_r\n -\tstruct reftable_record got_rec = { NULL };\n +\tstruct reftable_record got = {\n +\t\t.type = BLOCK_TYPE_OBJ,\n -+\t\t.obj = { 0 },\n ++\t\t.u.obj = { 0 },\n +\t};\n \tint err = 0;\n \tstruct indexed_table_ref_iter *itr = NULL;\n @@ reftable/reader.c: static int reftable_reader_refs_for_indexed(struct reftable_r\n \n -\tif (err > 0 ||\n -\t memcmp(want.hash_prefix, got.hash_prefix, r->object_id_len)) {\n -+\tif (err > 0 || memcmp(want.obj.hash_prefix, got.obj.hash_prefix,\n ++\tif (err > 0 || memcmp(want.u.obj.hash_prefix, got.u.obj.hash_prefix,\n +\t\t\t r->object_id_len)) {\n \t\t/* didn't find it; return empty iterator */\n \t\titerator_set_empty(it);\n @@ reftable/reader.c: static int reftable_reader_refs_for_indexed(struct reftable_r\n \n \terr = new_indexed_table_ref_iter(&itr, r, oid, hash_size(r->hash_id),\n -\t\t\t\t\t got.offsets, got.offset_len);\n -+\t\t\t\t\t got.obj.offsets, got.obj.offset_len);\n ++\t\t\t\t\t got.u.obj.offsets, got.u.obj.offset_len);\n \tif (err < 0)\n \t\tgoto done;\n -\tgot.offsets = NULL;\n -+\tgot.obj.offsets = NULL;\n ++\tgot.u.obj.offsets = NULL;\n \titerator_from_indexed_table_ref_iter(it, itr);\n \n done:\n @@ reftable/record.c: void string_view_consume(struct string_view *s, int n)\n +{\n +\tswitch (rec->type) {\n +\tcase BLOCK_TYPE_REF:\n -+\t\treturn &rec->ref;\n ++\t\treturn &rec->u.ref;\n +\tcase BLOCK_TYPE_LOG:\n -+\t\treturn &rec->log;\n ++\t\treturn &rec->u.log;\n +\tcase BLOCK_TYPE_INDEX:\n -+\t\treturn &rec->idx;\n ++\t\treturn &rec->u.idx;\n +\tcase BLOCK_TYPE_OBJ:\n -+\t\treturn &rec->obj;\n ++\t\treturn &rec->u.obj;\n +\t}\n +\tabort();\n +}\n @@ reftable/record.c: void string_view_consume(struct string_view *s, int n)\n +{\n +\tstruct reftable_record clean_idx = {\n +\t\t.type = BLOCK_TYPE_INDEX,\n -+\t\t.idx = {\n ++\t\t.u.idx = {\n +\t\t\t.last_key = STRBUF_INIT,\n +\t\t},\n +\t};\n @@ reftable/record.h: struct reftable_obj_record {\n +\t\tstruct reftable_log_record log;\n +\t\tstruct reftable_obj_record obj;\n +\t\tstruct reftable_index_record idx;\n -+\t};\n ++\t} u;\n +};\n +\n /* see struct record_vtable */\n @@ reftable/record_test.c\n \tcase BLOCK_TYPE_REF:\n -\t\tEXPECT(reftable_ref_record_equal(reftable_record_as_ref(©),\n -\t\t\t\t\t\t reftable_record_as_ref(rec),\n -+\t\tEXPECT(reftable_ref_record_equal(©.ref, &rec->ref,\n ++\t\tEXPECT(reftable_ref_record_equal(©.u.ref, &rec->u.ref,\n \t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\tbreak;\n \tcase BLOCK_TYPE_LOG:\n -\t\tEXPECT(reftable_log_record_equal(reftable_record_as_log(©),\n -\t\t\t\t\t\t reftable_record_as_log(rec),\n -+\t\tEXPECT(reftable_log_record_equal(©.log, &rec->log,\n ++\t\tEXPECT(reftable_log_record_equal(©.u.log, &rec->u.log,\n \t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\tbreak;\n \t}\n @@ reftable/record_test.c: static void test_reftable_ref_record_roundtrip(void)\n \t\tint n, m;\n \n -\t\tin.value_type = i;\n -+\t\tin.ref.value_type = i;\n ++\t\tin.u.ref.value_type = i;\n \t\tswitch (i) {\n \t\tcase REFTABLE_REF_DELETION:\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_VAL1:\n -\t\t\tin.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n -\t\t\tset_hash(in.value.val1, 1);\n -+\t\t\tin.ref.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n -+\t\t\tset_hash(in.ref.value.val1, 1);\n ++\t\t\tin.u.ref.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n ++\t\t\tset_hash(in.u.ref.value.val1, 1);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_VAL2:\n -\t\t\tin.value.val2.value = reftable_malloc(GIT_SHA1_RAWSZ);\n -\t\t\tset_hash(in.value.val2.value, 1);\n -\t\t\tin.value.val2.target_value =\n -+\t\t\tin.ref.value.val2.value =\n ++\t\t\tin.u.ref.value.val2.value =\n +\t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n -+\t\t\tset_hash(in.ref.value.val2.value, 1);\n -+\t\t\tin.ref.value.val2.target_value =\n ++\t\t\tset_hash(in.u.ref.value.val2.value, 1);\n ++\t\t\tin.u.ref.value.val2.target_value =\n \t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n -\t\t\tset_hash(in.value.val2.target_value, 2);\n -+\t\t\tset_hash(in.ref.value.val2.target_value, 2);\n ++\t\t\tset_hash(in.u.ref.value.val2.target_value, 2);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_SYMREF:\n -\t\t\tin.value.symref = xstrdup(\"target\");\n -+\t\t\tin.ref.value.symref = xstrdup(\"target\");\n ++\t\t\tin.u.ref.value.symref = xstrdup(\"target\");\n \t\t\tbreak;\n \t\t}\n -\t\tin.refname = xstrdup(\"refs/heads/master\");\n -+\t\tin.ref.refname = xstrdup(\"refs/heads/master\");\n ++\t\tin.u.ref.refname = xstrdup(\"refs/heads/master\");\n \n -\t\treftable_record_from_ref(&rec, &in);\n -\t\ttest_copy(&rec);\n @@ reftable/record_test.c: static void test_reftable_ref_record_roundtrip(void)\n \n -\t\tEXPECT(reftable_ref_record_equal(&in, &out, GIT_SHA1_RAWSZ));\n -\t\treftable_record_release(&rec_out);\n -+\t\tEXPECT(reftable_ref_record_equal(&in.ref, &out.ref,\n ++\t\tEXPECT(reftable_ref_record_equal(&in.u.ref, &out.u.ref,\n +\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n +\t\treftable_record_release(&in);\n \n @@ reftable/record_test.c: static void test_reftable_log_record_roundtrip(void)\n \t\t/* populate out, to check for leaks. */\n -\t\tstruct reftable_log_record out = {\n -\t\t\t.refname = xstrdup(\"old name\"),\n +-\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n +-\t\t\t.value = {\n +-\t\t\t\t.update = {\n +-\t\t\t\t\t.new_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n +-\t\t\t\t\t.old_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n +-\t\t\t\t\t.name = xstrdup(\"old name\"),\n +-\t\t\t\t\t.email = xstrdup(\"old@email\"),\n +-\t\t\t\t\t.message = xstrdup(\"old message\"),\n +\t\tstruct reftable_record out = {\n +\t\t\t.type = BLOCK_TYPE_LOG,\n -+\t\t\t.log = {\n ++\t\t\t.u.log = {\n +\t\t\t\t.refname = xstrdup(\"old name\"),\n - \t\t\t.value_type = REFTABLE_LOG_UPDATE,\n - \t\t\t.value = {\n - \t\t\t\t.update = {\n -@@ reftable/record_test.c: static void test_reftable_log_record_roundtrip(void)\n - \t\t\t\t\t.message = xstrdup(\"old message\"),\n ++\t\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n ++\t\t\t\t.value = {\n ++\t\t\t\t\t.update = {\n ++\t\t\t\t\t\t.new_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n ++\t\t\t\t\t\t.old_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n ++\t\t\t\t\t\t.name = xstrdup(\"old name\"),\n ++\t\t\t\t\t\t.email = xstrdup(\"old@email\"),\n ++\t\t\t\t\t\t.message = xstrdup(\"old message\"),\n ++\t\t\t\t\t},\n \t\t\t\t},\n \t\t\t},\n -+\t\t\t},\n \t\t};\n -\t\tstruct reftable_record rec_out = { NULL };\n \t\tint n, m, valtype;\n \n -\t\treftable_record_from_log(&rec, &in[i]);\n -+\t\trec.log = in[i];\n ++\t\trec.u.log = in[i];\n \n \t\ttest_copy(&rec);\n \n @@ reftable/record_test.c: static void test_reftable_log_record_roundtrip(void)\n \t\tEXPECT(n == m);\n \n -\t\tEXPECT(reftable_log_record_equal(&in[i], &out, GIT_SHA1_RAWSZ));\n -+\t\tEXPECT(reftable_log_record_equal(&in[i], &out.log,\n ++\t\tEXPECT(reftable_log_record_equal(&in[i], &out.u.log,\n +\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\treftable_log_record_release(&in[i]);\n \t\tstrbuf_release(&key);\n @@ reftable/record_test.c: static void test_reftable_obj_record_roundtrip(void)\n -\t\tstruct reftable_record rec = { NULL };\n +\t\tstruct reftable_record in = {\n +\t\t\t.type = BLOCK_TYPE_OBJ,\n -+\t\t\t.obj = recs[i],\n ++\t\t\t.u.obj = recs[i],\n +\t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n -\t\tstruct reftable_obj_record out = { NULL };\n @@ reftable/record_test.c: static void test_reftable_obj_record_roundtrip(void)\n \n -\t\tEXPECT(in.hash_prefix_len == out.hash_prefix_len);\n -\t\tEXPECT(in.offset_len == out.offset_len);\n -+\t\tEXPECT(in.obj.hash_prefix_len == out.obj.hash_prefix_len);\n -+\t\tEXPECT(in.obj.offset_len == out.obj.offset_len);\n - \n +-\n -\t\tEXPECT(!memcmp(in.hash_prefix, out.hash_prefix,\n -\t\t\t in.hash_prefix_len));\n -\t\tEXPECT(0 == memcmp(in.offsets, out.offsets,\n -\t\t\t\t sizeof(uint64_t) * in.offset_len));\n -+\t\tEXPECT(!memcmp(in.obj.hash_prefix, out.obj.hash_prefix,\n -+\t\t\t in.obj.hash_prefix_len));\n -+\t\tEXPECT(0 == memcmp(in.obj.offsets, out.obj.offsets,\n -+\t\t\t\t sizeof(uint64_t) * in.obj.offset_len));\n ++\t\tEXPECT(in.u.obj.hash_prefix_len == out.u.obj.hash_prefix_len);\n ++\t\tEXPECT(in.u.obj.offset_len == out.u.obj.offset_len);\n ++\t\tif (in.u.obj.hash_prefix_len)\n ++\t\t\tEXPECT(!memcmp(in.u.obj.hash_prefix, out.u.obj.hash_prefix,\n ++\t\t\t\t in.u.obj.hash_prefix_len));\n ++\t\tif (in.u.obj.offset_len)\n ++\t\t\tEXPECT(!memcmp(in.u.obj.offsets, out.u.obj.offsets,\n ++\t\t\t\t\t sizeof(uint64_t) * in.u.obj.offset_len));\n \t\tstrbuf_release(&key);\n -\t\treftable_record_release(&rec_out);\n +\t\treftable_record_release(&out);\n @@ reftable/record_test.c: static void test_reftable_obj_record_roundtrip(void)\n -\t\t.last_key = STRBUF_INIT,\n +\tstruct reftable_record in = {\n +\t\t.type = BLOCK_TYPE_INDEX,\n -+\t\t.idx = {\n ++\t\t.u.idx = {\n +\t\t\t.offset = 42,\n +\t\t\t.last_key = STRBUF_INIT,\n +\t\t},\n @@ reftable/record_test.c: static void test_reftable_index_record_roundtrip(void)\n -\tstruct reftable_record out_rec = { NULL };\n +\tstruct reftable_record out = {\n +\t\t.type = BLOCK_TYPE_INDEX,\n -+\t\t.idx = { .last_key = STRBUF_INIT },\n ++\t\t.u.idx = { .last_key = STRBUF_INIT },\n +\t};\n \tint n, m;\n \tuint8_t extra;\n @@ reftable/record_test.c: static void test_reftable_index_record_roundtrip(void)\n -\treftable_record_from_index(&rec, &in);\n -\treftable_record_key(&rec, &key);\n -\ttest_copy(&rec);\n -+\tstrbuf_addstr(&in.idx.last_key, \"refs/heads/master\");\n ++\tstrbuf_addstr(&in.u.idx.last_key, \"refs/heads/master\");\n +\treftable_record_key(&in, &key);\n +\ttest_copy(&in);\n \n -\tEXPECT(0 == strbuf_cmp(&key, &in.last_key));\n -\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n -+\tEXPECT(0 == strbuf_cmp(&key, &in.idx.last_key));\n ++\tEXPECT(0 == strbuf_cmp(&key, &in.u.idx.last_key));\n +\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(n > 0);\n \n @@ reftable/record_test.c: static void test_reftable_index_record_roundtrip(void)\n \tEXPECT(m == n);\n \n -\tEXPECT(in.offset == out.offset);\n -+\tEXPECT(in.idx.offset == out.idx.offset);\n ++\tEXPECT(in.u.idx.offset == out.u.idx.offset);\n \n -\treftable_record_release(&out_rec);\n +\treftable_record_release(&out);\n \tstrbuf_release(&key);\n -\tstrbuf_release(&in.last_key);\n -+\tstrbuf_release(&in.idx.last_key);\n ++\tstrbuf_release(&in.u.idx.last_key);\n }\n \n int record_test_main(int argc, const char *argv[])\n @@ reftable/writer.c: done:\n -\tstruct reftable_ref_record copy = *ref;\n +\tstruct reftable_record rec = {\n +\t\t.type = BLOCK_TYPE_REF,\n -+\t\t.ref = *ref,\n ++\t\t.u.ref = *ref,\n +\t};\n \tint err = 0;\n \n @@ reftable/writer.c: int reftable_writer_add_ref(struct reftable_writer *w,\n \n -\treftable_record_from_ref(&rec, ©);\n -\tcopy.update_index -= w->min_update_index;\n -+\trec.ref.update_index -= w->min_update_index;\n ++\trec.u.ref.update_index -= w->min_update_index;\n \n \terr = writer_add_record(w, &rec);\n \tif (err < 0)\n @@ reftable/writer.c: int reftable_writer_add_refs(struct reftable_writer *w,\n -\tstruct reftable_record rec = { NULL };\n +\tstruct reftable_record rec = {\n +\t\t.type = BLOCK_TYPE_LOG,\n -+\t\t.log = *log,\n ++\t\t.u.log = *log,\n +\t};\n \tif (w->block_writer &&\n \t block_writer_type(w->block_writer) == BLOCK_TYPE_REF) {\n @@ reftable/writer.c: static int writer_finish_section(struct reftable_writer *w)\n -\t\t\treftable_record_from_index(&rec, idx + i);\n +\t\t\tstruct reftable_record rec = {\n +\t\t\t\t.type = BLOCK_TYPE_INDEX,\n -+\t\t\t\t.idx = idx[i],\n ++\t\t\t\t.u.idx = idx[i],\n +\t\t\t};\n \t\t\tif (block_writer_add(w->block_writer, &rec) == 0) {\n \t\t\t\tcontinue;\n @@ reftable/writer.c: static void write_object_record(void *void_arg, void *key)\n -\t\t.offset_len = entry->offset_len,\n +\tstruct reftable_record rec = {\n +\t\t.type = BLOCK_TYPE_OBJ,\n -+\t\t.obj = {\n ++\t\t.u.obj = {\n +\t\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n +\t\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n +\t\t\t.offsets = entry->offsets,\n @@ reftable/writer.c: static void write_object_record(void *void_arg, void *key)\n \t\tgoto done;\n -\tobj_rec.offset_len = 0;\n +\n -+\trec.obj.offset_len = 0;\n ++\trec.u.obj.offset_len = 0;\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \n \t/* Should be able to write into a fresh block. */\n\n-- \ngitgitgadget\n"},{"id":"443522","messageId":"1ddcfe61ebc2c4ccb5fa82380ac00a62bf3aebdf.1639000187.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v2.git.git.1639000187.gitgitgadget@gmail.com","subject":"[PATCH v2 02/11] reftable: fix resource leak in error path","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-08T21:49:37Z","receivedAt":"2021-12-08T21:49:55Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis would be triggered by corrupt files, so it doesn't have test coverage. This\nwas discovered by a Coverity scan.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/block.c | 27 +++++++++++++++++++--------\n 1 file changed, 19 insertions(+), 8 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 855e3f5c947..79b6f3aac72 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -188,13 +188,16 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tuint32_t full_block_size = table_block_size;\n \tuint8_t typ = block->data[header_off];\n \tuint32_t sz = get_be24(block->data + header_off + 1);\n-\n+\tint err = 0;\n \tuint16_t restart_count = 0;\n \tuint32_t restart_start = 0;\n \tuint8_t *restart_bytes = NULL;\n+\tuint8_t *uncompressed = NULL;\n \n-\tif (!reftable_is_block_type(typ))\n-\t\treturn REFTABLE_FORMAT_ERROR;\n+\tif (!reftable_is_block_type(typ)) {\n+\t\terr = REFTABLE_FORMAT_ERROR;\n+\t\tgoto done;\n+\t}\n \n \tif (typ == BLOCK_TYPE_LOG) {\n \t\tint block_header_skip = 4 + header_off;\n@@ -203,7 +206,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\tuLongf src_len = block->len - block_header_skip;\n \t\t/* Log blocks specify the *uncompressed* size in their header.\n \t\t */\n-\t\tuint8_t *uncompressed = reftable_malloc(sz);\n+\t\tuncompressed = reftable_malloc(sz);\n \n \t\t/* Copy over the block header verbatim. It's not compressed. */\n \t\tmemcpy(uncompressed, block->data, block_header_skip);\n@@ -213,15 +216,19 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\t uncompress2(uncompressed + block_header_skip, &dst_len,\n \t\t\t\tblock->data + block_header_skip, &src_len)) {\n \t\t\treftable_free(uncompressed);\n-\t\t\treturn REFTABLE_ZLIB_ERROR;\n+\t\t\terr = REFTABLE_ZLIB_ERROR;\n+\t\t\tgoto done;\n \t\t}\n \n-\t\tif (dst_len + block_header_skip != sz)\n-\t\t\treturn REFTABLE_FORMAT_ERROR;\n+\t\tif (dst_len + block_header_skip != sz) {\n+\t\t\terr = REFTABLE_FORMAT_ERROR;\n+\t\t\tgoto done;\n+\t\t}\n \n \t\t/* We're done with the input data. */\n \t\treftable_block_done(block);\n \t\tblock->data = uncompressed;\n+\t\tuncompressed = NULL;\n \t\tblock->len = sz;\n \t\tblock->source = malloc_block_source();\n \t\tfull_block_size = src_len + block_header_skip;\n@@ -251,7 +258,11 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tbr->restart_count = restart_count;\n \tbr->restart_bytes = restart_bytes;\n \n-\treturn 0;\n+done:\n+\tif (uncompressed) {\n+\t\treftable_free(uncompressed);\n+\t}\n+\treturn err;\n }\n \n static uint32_t block_reader_restart_offset(struct block_reader *br, int i)\n-- \ngitgitgadget\n\n"},{"id":"443524","messageId":"e052b2a61d618a6af0f586505d2a7a95d4ed4492.1639000187.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v2.git.git.1639000187.gitgitgadget@gmail.com","subject":"[PATCH v2 03/11] reftable: fix resource leak blocksource.c","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-08T21:49:38Z","receivedAt":"2021-12-08T21:49:56Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis would be triggered in the unlikely event of fstat() failing on an opened\nfile.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/blocksource.c | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/reftable/blocksource.c b/reftable/blocksource.c\nindex 0044eecd9aa..2605371c28d 100644\n--- a/reftable/blocksource.c\n+++ b/reftable/blocksource.c\n@@ -134,8 +134,10 @@ int reftable_block_source_from_file(struct reftable_block_source *bs,\n \t}\n \n \terr = fstat(fd, &st);\n-\tif (err < 0)\n-\t\treturn -1;\n+\tif (err < 0) {\n+\t\tclose(fd);\n+\t\treturn REFTABLE_IO_ERROR;\n+\t}\n \n \tp = reftable_calloc(sizeof(struct file_block_source));\n \tp->size = st.st_size;\n-- \ngitgitgadget\n\n"},{"id":"443525","messageId":"9063137457be97352e99538808c914339b2e93c6.1639000187.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v2.git.git.1639000187.gitgitgadget@gmail.com","subject":"[PATCH v2 04/11] reftable: check reftable_stack_auto_compact() return value","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-08T21:49:39Z","receivedAt":"2021-12-08T21:49:59Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nFixes a problem detected by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack_test.c | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/reftable/stack_test.c b/reftable/stack_test.c\nindex eb0b7228b0c..d628420e63a 100644\n--- a/reftable/stack_test.c\n+++ b/reftable/stack_test.c\n@@ -814,6 +814,7 @@ static void test_reftable_stack_auto_compaction(void)\n \t\tEXPECT_ERR(err);\n \n \t\terr = reftable_stack_auto_compact(st);\n+\t\tEXPECT_ERR(err);\n \t\tEXPECT(i < 3 || st->merged->stack_len < 2 * fastlog2(i));\n \t}\n \n-- \ngitgitgadget\n\n"},{"id":"443526","messageId":"5020be156aeb5efe08c0377f15d8c27fa5c4d588.1639000187.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v2.git.git.1639000187.gitgitgadget@gmail.com","subject":"[PATCH v2 05/11] reftable: ignore remove() return value in stack_test.c","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-08T21:49:40Z","receivedAt":"2021-12-08T21:50:04Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nIf the cleanup fails, there is nothing we can do.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack_test.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/reftable/stack_test.c b/reftable/stack_test.c\nindex d628420e63a..4b7292945c3 100644\n--- a/reftable/stack_test.c\n+++ b/reftable/stack_test.c\n@@ -89,7 +89,7 @@ static void test_read_file(void)\n \t\tEXPECT(0 == strcmp(want[i], names[i]));\n \t}\n \tfree_names(names);\n-\tremove(fn);\n+\t(void) remove(fn);\n }\n \n static void test_parse_names(void)\n-- \ngitgitgadget\n\n"},{"id":"443527","messageId":"64c18d01cad70545e43c6a8e8d0547881051055b.1639000187.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v2.git.git.1639000187.gitgitgadget@gmail.com","subject":"[PATCH v2 06/11] reftable: fix resource warning","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-08T21:49:41Z","receivedAt":"2021-12-08T21:50:05Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis would trigger in the unlikely event that we are compacting, and the next\navailable file handle is 0.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack.c | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/reftable/stack.c b/reftable/stack.c\nindex df5021ebf08..10dfd370e8e 100644\n--- a/reftable/stack.c\n+++ b/reftable/stack.c\n@@ -877,7 +877,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \tstruct strbuf new_table_path = STRBUF_INIT;\n \tint err = 0;\n \tint have_lock = 0;\n-\tint lock_file_fd = 0;\n+\tint lock_file_fd = -1;\n \tint compact_count = last - first + 1;\n \tchar **listp = NULL;\n \tchar **delete_on_success =\n@@ -911,7 +911,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \t}\n \t/* Don't want to write to the lock for now. */\n \tclose(lock_file_fd);\n-\tlock_file_fd = 0;\n+\tlock_file_fd = -1;\n \n \thave_lock = 1;\n \terr = stack_uptodate(st);\n@@ -932,7 +932,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \n \t\tsublock_file_fd = open(subtab_lock.buf,\n \t\t\t\t O_EXCL | O_CREAT | O_WRONLY, 0644);\n-\t\tif (sublock_file_fd > 0) {\n+\t\tif (sublock_file_fd >= 0) {\n \t\t\tclose(sublock_file_fd);\n \t\t} else if (sublock_file_fd < 0) {\n \t\t\tif (errno == EEXIST) {\n@@ -1013,7 +1013,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \t\tgoto done;\n \t}\n \terr = close(lock_file_fd);\n-\tlock_file_fd = 0;\n+\tlock_file_fd = -1;\n \tif (err < 0) {\n \t\terr = REFTABLE_IO_ERROR;\n \t\tunlink(new_table_path.buf);\n@@ -1050,9 +1050,9 @@ done:\n \t\tlistp++;\n \t}\n \tfree_names(subtable_locks);\n-\tif (lock_file_fd > 0) {\n+\tif (lock_file_fd >= 0) {\n \t\tclose(lock_file_fd);\n-\t\tlock_file_fd = 0;\n+\t\tlock_file_fd = -1;\n \t}\n \tif (have_lock) {\n \t\tunlink(lock_file_name.buf);\n-- \ngitgitgadget\n\n"},{"id":"443528","messageId":"700387ac5d386aec5c1525223bd5952b037b15cf.1639000187.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v2.git.git.1639000187.gitgitgadget@gmail.com","subject":"[PATCH v2 07/11] reftable: fix NULL derefs in error paths","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-08T21:49:42Z","receivedAt":"2021-12-08T21:50:06Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSpotted by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/reader.c | 3 ++-\n reftable/stack.c | 3 ++-\n 2 files changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 006709a645a..8d308d858f8 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -796,6 +796,7 @@ int reftable_reader_print_file(const char *tablename)\n \treftable_table_from_reader(&tab, r);\n \terr = reftable_table_print(&tab);\n done:\n-\treftable_reader_free(r);\n+\tif (r)\n+\t\treftable_reader_free(r);\n \treturn err;\n }\ndiff --git a/reftable/stack.c b/reftable/stack.c\nindex 10dfd370e8e..eb03b6c74f6 100644\n--- a/reftable/stack.c\n+++ b/reftable/stack.c\n@@ -707,7 +707,8 @@ done:\n \tstrbuf_release(&temp_tab_file_name);\n \tstrbuf_release(&tab_file_name);\n \tstrbuf_release(&next_name);\n-\treftable_writer_free(wr);\n+\tif (wr)\n+\t\treftable_writer_free(wr);\n \treturn err;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"443529","messageId":"713f1d09f6837989e57026cffcaffea755d6a3f7.1639000187.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v2.git.git.1639000187.gitgitgadget@gmail.com","subject":"[PATCH v2 08/11] reftable: order unittests by complexity","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-08T21:49:43Z","receivedAt":"2021-12-08T21:50:09Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis is a more pratical ordering when working on refactorings of the reftable\ncode.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n t/helper/test-reftable.c | 9 +++++----\n 1 file changed, 5 insertions(+), 4 deletions(-)\n\ndiff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c\nindex 26b03d7b789..f08d66df949 100644\n--- a/t/helper/test-reftable.c\n+++ b/t/helper/test-reftable.c\n@@ -3,15 +3,16 @@\n \n int cmd__reftable(int argc, const char **argv)\n {\n+\t// test from simple to complex.\n \tbasics_test_main(argc, argv);\n+\trecord_test_main(argc, argv);\n \tblock_test_main(argc, argv);\n-\tmerged_test_main(argc, argv);\n+\ttree_test_main(argc, argv);\n \tpq_test_main(argc, argv);\n-\trecord_test_main(argc, argv);\n-\trefname_test_main(argc, argv);\n \treadwrite_test_main(argc, argv);\n+\tmerged_test_main(argc, argv);\n \tstack_test_main(argc, argv);\n-\ttree_test_main(argc, argv);\n+\trefname_test_main(argc, argv);\n \treturn 0;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"443530","messageId":"cb601b51a47efa7df019e386c2ea050b72071604.1639000187.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v2.git.git.1639000187.gitgitgadget@gmail.com","subject":"[PATCH v2 09/11] reftable: drop stray printf in readwrite_test","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-08T21:49:44Z","receivedAt":"2021-12-08T21:50:13Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/readwrite_test.c | 1 -\n 1 file changed, 1 deletion(-)\n\ndiff --git a/reftable/readwrite_test.c b/reftable/readwrite_test.c\nindex 5f6bcc2f775..7c40b9b77f3 100644\n--- a/reftable/readwrite_test.c\n+++ b/reftable/readwrite_test.c\n@@ -597,7 +597,6 @@ static void test_write_key_order(void)\n \terr = reftable_writer_add_ref(w, &refs[0]);\n \tEXPECT_ERR(err);\n \terr = reftable_writer_add_ref(w, &refs[1]);\n-\tprintf(\"%d\\n\", err);\n \tEXPECT(err == REFTABLE_API_ERROR);\n \treftable_writer_close(w);\n \treftable_writer_free(w);\n-- \ngitgitgadget\n\n"},{"id":"443531","messageId":"a0f83eff19f0e594601c96971f5968fbf4543536.1639000187.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v2.git.git.1639000187.gitgitgadget@gmail.com","subject":"[PATCH v2 10/11] reftable: handle null refnames in reftable_ref_record_equal","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-08T21:49:45Z","receivedAt":"2021-12-08T21:50:14Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSpotted by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 8536bd03aa9..8bbcbff1e69 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -1154,9 +1154,11 @@ int reftable_ref_record_equal(struct reftable_ref_record *a,\n \t\t\t struct reftable_ref_record *b, int hash_size)\n {\n \tassert(hash_size > 0);\n-\tif (!(0 == strcmp(a->refname, b->refname) &&\n-\t a->update_index == b->update_index &&\n-\t a->value_type == b->value_type))\n+\tif (!null_streq(a->refname, b->refname))\n+\t\treturn 0;\n+\n+\tif (a->update_index != b->update_index ||\n+\t a->value_type != b->value_type)\n \t\treturn 0;\n \n \tswitch (a->value_type) {\n-- \ngitgitgadget\n\n"},{"id":"443532","messageId":"a2743033cfd9c817f5f8544491425c5c79c4434c.1639000187.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v2.git.git.1639000187.gitgitgadget@gmail.com","subject":"[PATCH v2 11/11] reftable: make reftable_record a tagged union","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-08T21:49:46Z","receivedAt":"2021-12-08T21:50:15Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis reduces the amount of glue code, because we don't need a void pointer or\nvtable within the structure.\n\nThe only snag is that reftable_index_record contain a strbuf, so it cannot be\nzero-initialized. To address this, introduce reftable_record_for() to create a\nfresh instance, given a record type.\n\nThanks to Peff for the suggestion.\n\nHelped-by: Jeff King \nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/block.c | 4 +-\n reftable/block_test.c | 22 ++---\n reftable/generic.c | 35 ++++----\n reftable/iter.c | 4 +-\n reftable/merged.c | 37 +++++----\n reftable/pq.c | 3 +-\n reftable/pq_test.c | 31 +++----\n reftable/reader.c | 105 ++++++++++++------------\n reftable/record.c | 176 +++++++++++++++------------------------\n reftable/record.h | 45 +++++-----\n reftable/record_test.c | 181 +++++++++++++++++++++--------------------\n reftable/writer.c | 46 ++++++-----\n 12 files changed, 329 insertions(+), 360 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 79b6f3aac72..0bc4a5d84b3 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -382,7 +382,7 @@ int block_reader_seek(struct block_reader *br, struct block_iter *it,\n \t\t.key = *want,\n \t\t.r = br,\n \t};\n-\tstruct reftable_record rec = reftable_new_record(block_reader_type(br));\n+\tstruct reftable_record rec = reftable_record_for(block_reader_type(br));\n \tstruct strbuf key = STRBUF_INIT;\n \tint err = 0;\n \tstruct block_iter next = {\n@@ -424,7 +424,7 @@ int block_reader_seek(struct block_reader *br, struct block_iter *it,\n done:\n \tstrbuf_release(&key);\n \tstrbuf_release(&next.last_key);\n-\treftable_record_destroy(&rec);\n+\treftable_record_release(&rec);\n \n \treturn err;\n }\ndiff --git a/reftable/block_test.c b/reftable/block_test.c\nindex 4b3ea262dcb..fa2ee092ec0 100644\n--- a/reftable/block_test.c\n+++ b/reftable/block_test.c\n@@ -26,8 +26,9 @@ static void test_block_read_write(void)\n \tstruct block_writer bw = {\n \t\t.last_key = STRBUF_INIT,\n \t};\n-\tstruct reftable_ref_record ref = { NULL };\n-\tstruct reftable_record rec = { NULL };\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t};\n \tint i = 0;\n \tint n;\n \tstruct block_reader br = { 0 };\n@@ -40,7 +41,6 @@ static void test_block_read_write(void)\n \tblock.source = malloc_block_source();\n \tblock_writer_init(&bw, BLOCK_TYPE_REF, block.data, block_size,\n \t\t\t header_off, hash_size(GIT_SHA1_FORMAT_ID));\n-\treftable_record_from_ref(&rec, &ref);\n \n \tfor (i = 0; i < N; i++) {\n \t\tchar name[100];\n@@ -48,14 +48,14 @@ static void test_block_read_write(void)\n \t\tsnprintf(name, sizeof(name), \"branch%02d\", i);\n \t\tmemset(hash, i, sizeof(hash));\n \n-\t\tref.refname = name;\n-\t\tref.value_type = REFTABLE_REF_VAL1;\n-\t\tref.value.val1 = hash;\n+\t\trec.u.ref.refname = name;\n+\t\trec.u.ref.value_type = REFTABLE_REF_VAL1;\n+\t\trec.u.ref.value.val1 = hash;\n \n \t\tnames[i] = xstrdup(name);\n \t\tn = block_writer_add(&bw, &rec);\n-\t\tref.refname = NULL;\n-\t\tref.value_type = REFTABLE_REF_DELETION;\n+\t\trec.u.ref.refname = NULL;\n+\t\trec.u.ref.value_type = REFTABLE_REF_DELETION;\n \t\tEXPECT(n == 0);\n \t}\n \n@@ -74,7 +74,7 @@ static void test_block_read_write(void)\n \t\tif (r > 0) {\n \t\t\tbreak;\n \t\t}\n-\t\tEXPECT_STREQ(names[j], ref.refname);\n+\t\tEXPECT_STREQ(names[j], rec.u.ref.refname);\n \t\tj++;\n \t}\n \n@@ -92,7 +92,7 @@ static void test_block_read_write(void)\n \t\tn = block_iter_next(&it, &rec);\n \t\tEXPECT(n == 0);\n \n-\t\tEXPECT_STREQ(names[i], ref.refname);\n+\t\tEXPECT_STREQ(names[i], rec.u.ref.refname);\n \n \t\twant.len--;\n \t\tn = block_reader_seek(&br, &it, &want);\n@@ -100,7 +100,7 @@ static void test_block_read_write(void)\n \n \t\tn = block_iter_next(&it, &rec);\n \t\tEXPECT(n == 0);\n-\t\tEXPECT_STREQ(names[10 * (i / 10)], ref.refname);\n+\t\tEXPECT_STREQ(names[10 * (i / 10)], rec.u.ref.refname);\n \n \t\tblock_iter_close(&it);\n \t}\ndiff --git a/reftable/generic.c b/reftable/generic.c\nindex 7a8a738d860..4446b8ed360 100644\n--- a/reftable/generic.c\n+++ b/reftable/generic.c\n@@ -7,6 +7,7 @@ https://developers.google.com/open-source/licenses/bsd\n */\n \n #include \"basics.h\"\n+#include \"constants.h\"\n #include \"record.h\"\n #include \"generic.h\"\n #include \"reftable-iterator.h\"\n@@ -15,23 +16,21 @@ https://developers.google.com/open-source/licenses/bsd\n int reftable_table_seek_ref(struct reftable_table *tab,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_REF,\n+\t\t\t\t .u.ref = {\n+\t\t\t\t\t .refname = (char *)name,\n+\t\t\t\t } };\n \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n }\n \n int reftable_table_seek_log(struct reftable_table *tab,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = ~((uint64_t)0),\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n+\t\t\t\t .u.log = {\n+\t\t\t\t\t .refname = (char *)name,\n+\t\t\t\t\t .update_index = ~((uint64_t)0),\n+\t\t\t\t } };\n \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n }\n \n@@ -129,17 +128,19 @@ void reftable_iterator_destroy(struct reftable_iterator *it)\n int reftable_iterator_next_ref(struct reftable_iterator *it,\n \t\t\t struct reftable_ref_record *ref)\n {\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, ref);\n-\treturn iterator_next(it, &rec);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_REF };\n+\tint err = iterator_next(it, &rec);\n+\t*ref = rec.u.ref;\n+\treturn err;\n }\n \n int reftable_iterator_next_log(struct reftable_iterator *it,\n \t\t\t struct reftable_log_record *log)\n {\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, log);\n-\treturn iterator_next(it, &rec);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG };\n+\tint err = iterator_next(it, &rec);\n+\t*log = rec.u.log;\n+\treturn err;\n }\n \n int iterator_next(struct reftable_iterator *it, struct reftable_record *rec)\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex 93d04f735b8..a8d174c0406 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -32,7 +32,7 @@ static int filtering_ref_iterator_next(void *iter_arg,\n \t\t\t\t struct reftable_record *rec)\n {\n \tstruct filtering_ref_iterator *fri = iter_arg;\n-\tstruct reftable_ref_record *ref = rec->data;\n+\tstruct reftable_ref_record *ref = &rec->u.ref;\n \tint err = 0;\n \twhile (1) {\n \t\terr = reftable_iterator_next_ref(&fri->it, ref);\n@@ -127,7 +127,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n static int indexed_table_ref_iter_next(void *p, struct reftable_record *rec)\n {\n \tstruct indexed_table_ref_iter *it = p;\n-\tstruct reftable_ref_record *ref = rec->data;\n+\tstruct reftable_ref_record *ref = &rec->u.ref;\n \n \twhile (1) {\n \t\tint err = block_iter_next(&it->cur, rec);\ndiff --git a/reftable/merged.c b/reftable/merged.c\nindex e5b53da6db3..e9b0edec97d 100644\n--- a/reftable/merged.c\n+++ b/reftable/merged.c\n@@ -22,7 +22,7 @@ static int merged_iter_init(struct merged_iter *mi)\n {\n \tint i = 0;\n \tfor (i = 0; i < mi->stack_len; i++) {\n-\t\tstruct reftable_record rec = reftable_new_record(mi->typ);\n+\t\tstruct reftable_record rec = reftable_record_for(mi->typ);\n \t\tint err = iterator_next(&mi->stack[i], &rec);\n \t\tif (err < 0) {\n \t\t\treturn err;\n@@ -30,7 +30,7 @@ static int merged_iter_init(struct merged_iter *mi)\n \n \t\tif (err > 0) {\n \t\t\treftable_iterator_destroy(&mi->stack[i]);\n-\t\t\treftable_record_destroy(&rec);\n+\t\t\treftable_record_release(&rec);\n \t\t} else {\n \t\t\tstruct pq_entry e = {\n \t\t\t\t.rec = rec,\n@@ -57,18 +57,17 @@ static void merged_iter_close(void *p)\n static int merged_iter_advance_nonnull_subiter(struct merged_iter *mi,\n \t\t\t\t\t size_t idx)\n {\n-\tstruct reftable_record rec = reftable_new_record(mi->typ);\n \tstruct pq_entry e = {\n-\t\t.rec = rec,\n+\t\t.rec = reftable_record_for(mi->typ),\n \t\t.index = idx,\n \t};\n-\tint err = iterator_next(&mi->stack[idx], &rec);\n+\tint err = iterator_next(&mi->stack[idx], &e.rec);\n \tif (err < 0)\n \t\treturn err;\n \n \tif (err > 0) {\n \t\treftable_iterator_destroy(&mi->stack[idx]);\n-\t\treftable_record_destroy(&rec);\n+\t\treftable_record_release(&e.rec);\n \t\treturn 0;\n \t}\n \n@@ -126,11 +125,11 @@ static int merged_iter_next_entry(struct merged_iter *mi,\n \t\tif (err < 0) {\n \t\t\treturn err;\n \t\t}\n-\t\treftable_record_destroy(&top.rec);\n+\t\treftable_record_release(&top.rec);\n \t}\n \n \treftable_record_copy_from(rec, &entry.rec, hash_size(mi->hash_id));\n-\treftable_record_destroy(&entry.rec);\n+\treftable_record_release(&entry.rec);\n \tstrbuf_release(&entry_key);\n \treturn 0;\n }\n@@ -246,7 +245,7 @@ static int merged_table_seek_record(struct reftable_merged_table *mt,\n \t\tsizeof(struct reftable_iterator) * mt->stack_len);\n \tstruct merged_iter merged = {\n \t\t.stack = iters,\n-\t\t.typ = reftable_record_type(rec),\n+\t\t.typ = rec->type,\n \t\t.hash_id = mt->hash_id,\n \t\t.suppress_deletions = mt->suppress_deletions,\n \t};\n@@ -290,11 +289,12 @@ int reftable_merged_table_seek_ref(struct reftable_merged_table *mt,\n \t\t\t\t struct reftable_iterator *it,\n \t\t\t\t const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = {\n+\t\t\t.refname = (char *)name,\n+\t\t},\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n \treturn merged_table_seek_record(mt, it, &rec);\n }\n \n@@ -302,12 +302,13 @@ int reftable_merged_table_seek_log_at(struct reftable_merged_table *mt,\n \t\t\t\t struct reftable_iterator *it,\n \t\t\t\t const char *name, uint64_t update_index)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = update_index,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.u.log = {\n+\t\t\t.refname = (char *)name,\n+\t\t\t.update_index = update_index,\n+\t\t}\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n \treturn merged_table_seek_record(mt, it, &rec);\n }\n \ndiff --git a/reftable/pq.c b/reftable/pq.c\nindex efc474017a2..96ca6dd37b3 100644\n--- a/reftable/pq.c\n+++ b/reftable/pq.c\n@@ -74,6 +74,7 @@ struct pq_entry merged_iter_pqueue_remove(struct merged_iter_pqueue *pq)\n void merged_iter_pqueue_add(struct merged_iter_pqueue *pq, struct pq_entry e)\n {\n \tint i = 0;\n+\n \tif (pq->len == pq->cap) {\n \t\tpq->cap = 2 * pq->cap + 1;\n \t\tpq->heap = reftable_realloc(pq->heap,\n@@ -98,7 +99,7 @@ void merged_iter_pqueue_release(struct merged_iter_pqueue *pq)\n {\n \tint i = 0;\n \tfor (i = 0; i < pq->len; i++) {\n-\t\treftable_record_destroy(&pq->heap[i].rec);\n+\t\treftable_record_release(&pq->heap[i].rec);\n \t}\n \tFREE_AND_NULL(pq->heap);\n \tpq->len = pq->cap = 0;\ndiff --git a/reftable/pq_test.c b/reftable/pq_test.c\nindex c9bb05e37b7..9f04d84fed6 100644\n--- a/reftable/pq_test.c\n+++ b/reftable/pq_test.c\n@@ -31,7 +31,7 @@ static void test_pq(void)\n \tint N = ARRAY_SIZE(names) - 1;\n \n \tstruct merged_iter_pqueue pq = { NULL };\n-\tconst char *last = NULL;\n+\tchar *last = NULL;\n \n \tint i = 0;\n \tfor (i = 0; i < N; i++) {\n@@ -42,12 +42,14 @@ static void test_pq(void)\n \n \ti = 1;\n \tdo {\n-\t\tstruct reftable_record rec =\n-\t\t\treftable_new_record(BLOCK_TYPE_REF);\n-\t\tstruct pq_entry e = { 0 };\n-\n-\t\treftable_record_as_ref(&rec)->refname = names[i];\n-\t\te.rec = rec;\n+\t\tstruct pq_entry e = {\n+\t\t\t.rec = {\n+\t\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t\t.u.ref = {\n+\t\t\t\t\t.refname = names[i],\n+\t\t\t\t}\n+\t\t\t}\n+\t\t};\n \t\tmerged_iter_pqueue_add(&pq, e);\n \t\tmerged_iter_pqueue_check(pq);\n \t\ti = (i * 7) % N;\n@@ -55,19 +57,18 @@ static void test_pq(void)\n \n \twhile (!merged_iter_pqueue_is_empty(pq)) {\n \t\tstruct pq_entry e = merged_iter_pqueue_remove(&pq);\n-\t\tstruct reftable_ref_record *ref =\n-\t\t\treftable_record_as_ref(&e.rec);\n-\n+\t\tstruct reftable_record *rec = &e.rec;\n \t\tmerged_iter_pqueue_check(pq);\n \n+\t\tEXPECT(rec->type == BLOCK_TYPE_REF);\n \t\tif (last) {\n-\t\t\tEXPECT(strcmp(last, ref->refname) < 0);\n+\t\t\tEXPECT(strcmp(last, rec->u.ref.refname) < 0);\n \t\t}\n-\t\tlast = ref->refname;\n-\t\tref->refname = NULL;\n-\t\treftable_free(ref);\n+\t\t// this is names[i], so don't dealloc.\n+\t\tlast = rec->u.ref.refname;\n+\t\trec->u.ref.refname = NULL;\n+\t\treftable_record_release(rec);\n \t}\n-\n \tfor (i = 0; i < N; i++) {\n \t\treftable_free(names[i]);\n \t}\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 8d308d858f8..ef322791fda 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -238,9 +238,8 @@ static int table_iter_next_in_block(struct table_iter *ti,\n \t\t\t\t struct reftable_record *rec)\n {\n \tint res = block_iter_next(&ti->bi, rec);\n-\tif (res == 0 && reftable_record_type(rec) == BLOCK_TYPE_REF) {\n-\t\t((struct reftable_ref_record *)rec->data)->update_index +=\n-\t\t\tti->r->min_update_index;\n+\tif (res == 0 && rec->type == BLOCK_TYPE_REF) {\n+\t\trec->u.ref.update_index += ti->r->min_update_index;\n \t}\n \n \treturn res;\n@@ -345,7 +344,7 @@ static int table_iter_next_block(struct table_iter *dest,\n \n static int table_iter_next(struct table_iter *ti, struct reftable_record *rec)\n {\n-\tif (reftable_record_type(rec) != ti->typ)\n+\tif (rec->type != ti->typ)\n \t\treturn REFTABLE_API_ERROR;\n \n \twhile (1) {\n@@ -437,8 +436,7 @@ static int reader_start(struct reftable_reader *r, struct table_iter *ti,\n static int reader_seek_linear(struct reftable_reader *r, struct table_iter *ti,\n \t\t\t struct reftable_record *want)\n {\n-\tstruct reftable_record rec =\n-\t\treftable_new_record(reftable_record_type(want));\n+\tstruct reftable_record rec = reftable_record_for(want->type);\n \tstruct strbuf want_key = STRBUF_INIT;\n \tstruct strbuf got_key = STRBUF_INIT;\n \tstruct table_iter next = TABLE_ITER_INIT;\n@@ -475,7 +473,7 @@ static int reader_seek_linear(struct reftable_reader *r, struct table_iter *ti,\n \n done:\n \tblock_iter_close(&next.bi);\n-\treftable_record_destroy(&rec);\n+\treftable_record_release(&rec);\n \tstrbuf_release(&want_key);\n \tstrbuf_release(&got_key);\n \treturn err;\n@@ -485,38 +483,40 @@ static int reader_seek_indexed(struct reftable_reader *r,\n \t\t\t struct reftable_iterator *it,\n \t\t\t struct reftable_record *rec)\n {\n-\tstruct reftable_index_record want_index = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record want_index_rec = { NULL };\n-\tstruct reftable_index_record index_result = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record index_result_rec = { NULL };\n+\tstruct reftable_record want_index = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = { .last_key = STRBUF_INIT }\n+\t};\n+\tstruct reftable_record index_result = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = { .last_key = STRBUF_INIT },\n+\t};\n \tstruct table_iter index_iter = TABLE_ITER_INIT;\n \tstruct table_iter next = TABLE_ITER_INIT;\n \tint err = 0;\n \n-\treftable_record_key(rec, &want_index.last_key);\n-\treftable_record_from_index(&want_index_rec, &want_index);\n-\treftable_record_from_index(&index_result_rec, &index_result);\n-\n-\terr = reader_start(r, &index_iter, reftable_record_type(rec), 1);\n+\treftable_record_key(rec, &want_index.u.idx.last_key);\n+\terr = reader_start(r, &index_iter, rec->type, 1);\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = reader_seek_linear(r, &index_iter, &want_index_rec);\n+\terr = reader_seek_linear(r, &index_iter, &want_index);\n \twhile (1) {\n-\t\terr = table_iter_next(&index_iter, &index_result_rec);\n+\t\terr = table_iter_next(&index_iter, &index_result);\n \t\ttable_iter_block_done(&index_iter);\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = reader_table_iter_at(r, &next, index_result.offset, 0);\n+\t\terr = reader_table_iter_at(r, &next, index_result.u.idx.offset,\n+\t\t\t\t\t 0);\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = block_iter_seek(&next.bi, &want_index.last_key);\n+\t\terr = block_iter_seek(&next.bi, &want_index.u.idx.last_key);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n-\t\tif (next.typ == reftable_record_type(rec)) {\n+\t\tif (next.typ == rec->type) {\n \t\t\terr = 0;\n \t\t\tbreak;\n \t\t}\n@@ -540,8 +540,8 @@ static int reader_seek_indexed(struct reftable_reader *r,\n done:\n \tblock_iter_close(&next.bi);\n \ttable_iter_close(&index_iter);\n-\treftable_record_release(&want_index_rec);\n-\treftable_record_release(&index_result_rec);\n+\treftable_record_release(&want_index);\n+\treftable_record_release(&index_result);\n \treturn err;\n }\n \n@@ -549,15 +549,14 @@ static int reader_seek_internal(struct reftable_reader *r,\n \t\t\t\tstruct reftable_iterator *it,\n \t\t\t\tstruct reftable_record *rec)\n {\n-\tstruct reftable_reader_offsets *offs =\n-\t\treader_offsets_for(r, reftable_record_type(rec));\n+\tstruct reftable_reader_offsets *offs = reader_offsets_for(r, rec->type);\n \tuint64_t idx = offs->index_offset;\n \tstruct table_iter ti = TABLE_ITER_INIT;\n \tint err = 0;\n \tif (idx > 0)\n \t\treturn reader_seek_indexed(r, it, rec);\n \n-\terr = reader_start(r, &ti, reftable_record_type(rec), 0);\n+\terr = reader_start(r, &ti, rec->type, 0);\n \tif (err < 0)\n \t\treturn err;\n \terr = reader_seek_linear(r, &ti, rec);\n@@ -576,7 +575,7 @@ static int reader_seek_internal(struct reftable_reader *r,\n static int reader_seek(struct reftable_reader *r, struct reftable_iterator *it,\n \t\t struct reftable_record *rec)\n {\n-\tuint8_t typ = reftable_record_type(rec);\n+\tuint8_t typ = rec->type;\n \n \tstruct reftable_reader_offsets *offs = reader_offsets_for(r, typ);\n \tif (!offs->is_present) {\n@@ -590,11 +589,12 @@ static int reader_seek(struct reftable_reader *r, struct reftable_iterator *it,\n int reftable_reader_seek_ref(struct reftable_reader *r,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = {\n+\t\t\t.refname = (char *)name,\n+\t\t},\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n \treturn reader_seek(r, it, &rec);\n }\n \n@@ -602,12 +602,13 @@ int reftable_reader_seek_log_at(struct reftable_reader *r,\n \t\t\t\tstruct reftable_iterator *it, const char *name,\n \t\t\t\tuint64_t update_index)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = update_index,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.u.log = {\n+\t\t\t.refname = (char *)name,\n+\t\t\t.update_index = update_index,\n+\t\t}\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n \treturn reader_seek(r, it, &rec);\n }\n \n@@ -649,31 +650,33 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \t\t\t\t\t struct reftable_iterator *it,\n \t\t\t\t\t uint8_t *oid)\n {\n-\tstruct reftable_obj_record want = {\n-\t\t.hash_prefix = oid,\n-\t\t.hash_prefix_len = r->object_id_len,\n+\tstruct reftable_record want = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.u.obj = {\n+\t\t\t.hash_prefix = oid,\n+\t\t\t.hash_prefix_len = r->object_id_len,\n+\t\t},\n \t};\n-\tstruct reftable_record want_rec = { NULL };\n \tstruct reftable_iterator oit = { NULL };\n-\tstruct reftable_obj_record got = { NULL };\n-\tstruct reftable_record got_rec = { NULL };\n+\tstruct reftable_record got = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.u.obj = { 0 },\n+\t};\n \tint err = 0;\n \tstruct indexed_table_ref_iter *itr = NULL;\n \n \t/* Look through the reverse index. */\n-\treftable_record_from_obj(&want_rec, &want);\n-\terr = reader_seek(r, &oit, &want_rec);\n+\terr = reader_seek(r, &oit, &want);\n \tif (err != 0)\n \t\tgoto done;\n \n \t/* read out the reftable_obj_record */\n-\treftable_record_from_obj(&got_rec, &got);\n-\terr = iterator_next(&oit, &got_rec);\n+\terr = iterator_next(&oit, &got);\n \tif (err < 0)\n \t\tgoto done;\n \n-\tif (err > 0 ||\n-\t memcmp(want.hash_prefix, got.hash_prefix, r->object_id_len)) {\n+\tif (err > 0 || memcmp(want.u.obj.hash_prefix, got.u.obj.hash_prefix,\n+\t\t\t r->object_id_len)) {\n \t\t/* didn't find it; return empty iterator */\n \t\titerator_set_empty(it);\n \t\terr = 0;\n@@ -681,15 +684,15 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \t}\n \n \terr = new_indexed_table_ref_iter(&itr, r, oid, hash_size(r->hash_id),\n-\t\t\t\t\t got.offsets, got.offset_len);\n+\t\t\t\t\t got.u.obj.offsets, got.u.obj.offset_len);\n \tif (err < 0)\n \t\tgoto done;\n-\tgot.offsets = NULL;\n+\tgot.u.obj.offsets = NULL;\n \titerator_from_indexed_table_ref_iter(it, itr);\n \n done:\n \treftable_iterator_destroy(&oit);\n-\treftable_record_release(&got_rec);\n+\treftable_record_release(&got);\n \treturn err;\n }\n \ndiff --git a/reftable/record.c b/reftable/record.c\nindex 8bbcbff1e69..8cfd67b21e6 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -15,6 +15,10 @@ https://developers.google.com/open-source/licenses/bsd\n #include \"reftable-error.h\"\n #include \"basics.h\"\n \n+static struct reftable_record_vtable *\n+reftable_record_vtable(struct reftable_record *rec);\n+static void *reftable_record_data(struct reftable_record *rec);\n+\n int get_var_int(uint64_t *dest, struct string_view *in)\n {\n \tint ptr = 0;\n@@ -926,58 +930,6 @@ static struct reftable_record_vtable reftable_log_record_vtable = {\n \t.is_deletion = &reftable_log_record_is_deletion_void,\n };\n \n-struct reftable_record reftable_new_record(uint8_t typ)\n-{\n-\tstruct reftable_record rec = { NULL };\n-\tswitch (typ) {\n-\tcase BLOCK_TYPE_REF: {\n-\t\tstruct reftable_ref_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_ref_record));\n-\t\treftable_record_from_ref(&rec, r);\n-\t\treturn rec;\n-\t}\n-\n-\tcase BLOCK_TYPE_OBJ: {\n-\t\tstruct reftable_obj_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_obj_record));\n-\t\treftable_record_from_obj(&rec, r);\n-\t\treturn rec;\n-\t}\n-\tcase BLOCK_TYPE_LOG: {\n-\t\tstruct reftable_log_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_log_record));\n-\t\treftable_record_from_log(&rec, r);\n-\t\treturn rec;\n-\t}\n-\tcase BLOCK_TYPE_INDEX: {\n-\t\tstruct reftable_index_record empty = { .last_key =\n-\t\t\t\t\t\t\t STRBUF_INIT };\n-\t\tstruct reftable_index_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_index_record));\n-\t\t*r = empty;\n-\t\treftable_record_from_index(&rec, r);\n-\t\treturn rec;\n-\t}\n-\t}\n-\tabort();\n-\treturn rec;\n-}\n-\n-/* clear out the record, yielding the reftable_record data that was\n- * encapsulated. */\n-static void *reftable_record_yield(struct reftable_record *rec)\n-{\n-\tvoid *p = rec->data;\n-\trec->data = NULL;\n-\treturn p;\n-}\n-\n-void reftable_record_destroy(struct reftable_record *rec)\n-{\n-\treftable_record_release(rec);\n-\treftable_free(reftable_record_yield(rec));\n-}\n-\n static void reftable_index_record_key(const void *r, struct strbuf *dest)\n {\n \tconst struct reftable_index_record *rec = r;\n@@ -1055,91 +1007,47 @@ static struct reftable_record_vtable reftable_index_record_vtable = {\n \n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)\n {\n-\trec->ops->key(rec->data, dest);\n-}\n-\n-uint8_t reftable_record_type(struct reftable_record *rec)\n-{\n-\treturn rec->ops->type;\n+\treftable_record_vtable(rec)->key(reftable_record_data(rec), dest);\n }\n \n int reftable_record_encode(struct reftable_record *rec, struct string_view dest,\n \t\t\t int hash_size)\n {\n-\treturn rec->ops->encode(rec->data, dest, hash_size);\n+\treturn reftable_record_vtable(rec)->encode(reftable_record_data(rec),\n+\t\t\t\t\t\t dest, hash_size);\n }\n \n void reftable_record_copy_from(struct reftable_record *rec,\n \t\t\t struct reftable_record *src, int hash_size)\n {\n-\tassert(src->ops->type == rec->ops->type);\n+\tassert(src->type == rec->type);\n \n-\trec->ops->copy_from(rec->data, src->data, hash_size);\n+\treftable_record_vtable(rec)->copy_from(reftable_record_data(rec),\n+\t\t\t\t\t reftable_record_data(src),\n+\t\t\t\t\t hash_size);\n }\n \n uint8_t reftable_record_val_type(struct reftable_record *rec)\n {\n-\treturn rec->ops->val_type(rec->data);\n+\treturn reftable_record_vtable(rec)->val_type(reftable_record_data(rec));\n }\n \n int reftable_record_decode(struct reftable_record *rec, struct strbuf key,\n \t\t\t uint8_t extra, struct string_view src, int hash_size)\n {\n-\treturn rec->ops->decode(rec->data, key, extra, src, hash_size);\n+\treturn reftable_record_vtable(rec)->decode(reftable_record_data(rec),\n+\t\t\t\t\t\t key, extra, src, hash_size);\n }\n \n void reftable_record_release(struct reftable_record *rec)\n {\n-\trec->ops->release(rec->data);\n+\treftable_record_vtable(rec)->release(reftable_record_data(rec));\n }\n \n int reftable_record_is_deletion(struct reftable_record *rec)\n {\n-\treturn rec->ops->is_deletion(rec->data);\n-}\n-\n-void reftable_record_from_ref(struct reftable_record *rec,\n-\t\t\t struct reftable_ref_record *ref_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = ref_rec;\n-\trec->ops = &reftable_ref_record_vtable;\n-}\n-\n-void reftable_record_from_obj(struct reftable_record *rec,\n-\t\t\t struct reftable_obj_record *obj_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = obj_rec;\n-\trec->ops = &reftable_obj_record_vtable;\n-}\n-\n-void reftable_record_from_index(struct reftable_record *rec,\n-\t\t\t\tstruct reftable_index_record *index_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = index_rec;\n-\trec->ops = &reftable_index_record_vtable;\n-}\n-\n-void reftable_record_from_log(struct reftable_record *rec,\n-\t\t\t struct reftable_log_record *log_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = log_rec;\n-\trec->ops = &reftable_log_record_vtable;\n-}\n-\n-struct reftable_ref_record *reftable_record_as_ref(struct reftable_record *rec)\n-{\n-\tassert(reftable_record_type(rec) == BLOCK_TYPE_REF);\n-\treturn rec->data;\n-}\n-\n-struct reftable_log_record *reftable_record_as_log(struct reftable_record *rec)\n-{\n-\tassert(reftable_record_type(rec) == BLOCK_TYPE_LOG);\n-\treturn rec->data;\n+\treturn reftable_record_vtable(rec)->is_deletion(\n+\t\treftable_record_data(rec));\n }\n \n static int hash_equal(uint8_t *a, uint8_t *b, int hash_size)\n@@ -1212,3 +1120,53 @@ void string_view_consume(struct string_view *s, int n)\n \ts->buf += n;\n \ts->len -= n;\n }\n+\n+static void *reftable_record_data(struct reftable_record *rec)\n+{\n+\tswitch (rec->type) {\n+\tcase BLOCK_TYPE_REF:\n+\t\treturn &rec->u.ref;\n+\tcase BLOCK_TYPE_LOG:\n+\t\treturn &rec->u.log;\n+\tcase BLOCK_TYPE_INDEX:\n+\t\treturn &rec->u.idx;\n+\tcase BLOCK_TYPE_OBJ:\n+\t\treturn &rec->u.obj;\n+\t}\n+\tabort();\n+}\n+\n+static struct reftable_record_vtable *\n+reftable_record_vtable(struct reftable_record *rec)\n+{\n+\tswitch (rec->type) {\n+\tcase BLOCK_TYPE_REF:\n+\t\treturn &reftable_ref_record_vtable;\n+\tcase BLOCK_TYPE_LOG:\n+\t\treturn &reftable_log_record_vtable;\n+\tcase BLOCK_TYPE_INDEX:\n+\t\treturn &reftable_index_record_vtable;\n+\tcase BLOCK_TYPE_OBJ:\n+\t\treturn &reftable_obj_record_vtable;\n+\t}\n+\tabort();\n+}\n+\n+struct reftable_record reftable_record_for(uint8_t typ)\n+{\n+\tstruct reftable_record clean_idx = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = {\n+\t\t\t.last_key = STRBUF_INIT,\n+\t\t},\n+\t};\n+\tstruct reftable_record clean = {\n+\t\t.type = typ,\n+\t};\n+\n+\tif (typ == BLOCK_TYPE_INDEX) {\n+\t\treturn clean_idx;\n+\t}\n+\n+\treturn clean;\n+}\ndiff --git a/reftable/record.h b/reftable/record.h\nindex 498e8c50bf4..1fe0c14a198 100644\n--- a/reftable/record.h\n+++ b/reftable/record.h\n@@ -60,18 +60,9 @@ struct reftable_record_vtable {\n \tint (*is_deletion)(const void *rec);\n };\n \n-/* record is a generic wrapper for different types of records. */\n-struct reftable_record {\n-\tvoid *data;\n-\tstruct reftable_record_vtable *ops;\n-};\n-\n /* returns true for recognized block types. Block start with the block type. */\n int reftable_is_block_type(uint8_t typ);\n \n-/* creates a malloced record of the given type. Dispose with record_destroy */\n-struct reftable_record reftable_new_record(uint8_t typ);\n-\n /* Encode `key` into `dest`. Sets `is_restart` to indicate a restart. Returns\n * number of bytes written. */\n int reftable_encode_key(int *is_restart, struct string_view dest,\n@@ -97,8 +88,26 @@ struct reftable_obj_record {\n \tint offset_len;\n };\n \n+/* record is a generic wrapper for different types of records. It is normally\n+ * created on the stack, or embedded within another struct. If the type is\n+ * known, a fresh instance can be initialized explicitly. Otherwise, use\n+ * reftable_record_for() to initialize generically (as the index_record is not\n+ * valid as 0-initialized structure)\n+ */\n+struct reftable_record {\n+\tuint8_t type;\n+\tunion {\n+\t\tstruct reftable_ref_record ref;\n+\t\tstruct reftable_log_record log;\n+\t\tstruct reftable_obj_record obj;\n+\t\tstruct reftable_index_record idx;\n+\t} u;\n+};\n+\n /* see struct record_vtable */\n \n+/* return an initialized record for the given type */\n+struct reftable_record reftable_record_for(uint8_t typ);\n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest);\n uint8_t reftable_record_type(struct reftable_record *rec);\n void reftable_record_copy_from(struct reftable_record *rec,\n@@ -111,25 +120,9 @@ int reftable_record_decode(struct reftable_record *rec, struct strbuf key,\n \t\t\t int hash_size);\n int reftable_record_is_deletion(struct reftable_record *rec);\n \n-/* zeroes out the embedded record */\n+/* frees and zeroes out the embedded record */\n void reftable_record_release(struct reftable_record *rec);\n \n-/* clear and deallocate embedded record, and zero `rec`. */\n-void reftable_record_destroy(struct reftable_record *rec);\n-\n-/* initialize generic records from concrete records. The generic record should\n- * be zeroed out. */\n-void reftable_record_from_obj(struct reftable_record *rec,\n-\t\t\t struct reftable_obj_record *objrec);\n-void reftable_record_from_index(struct reftable_record *rec,\n-\t\t\t\tstruct reftable_index_record *idxrec);\n-void reftable_record_from_ref(struct reftable_record *rec,\n-\t\t\t struct reftable_ref_record *refrec);\n-void reftable_record_from_log(struct reftable_record *rec,\n-\t\t\t struct reftable_log_record *logrec);\n-struct reftable_ref_record *reftable_record_as_ref(struct reftable_record *ref);\n-struct reftable_log_record *reftable_record_as_log(struct reftable_record *ref);\n-\n /* for qsort. */\n int reftable_ref_record_compare_name(const void *a, const void *b);\n \ndiff --git a/reftable/record_test.c b/reftable/record_test.c\nindex f4ad7cace41..12161c2895a 100644\n--- a/reftable/record_test.c\n+++ b/reftable/record_test.c\n@@ -16,24 +16,22 @@\n \n static void test_copy(struct reftable_record *rec)\n {\n-\tstruct reftable_record copy =\n-\t\treftable_new_record(reftable_record_type(rec));\n+\tstruct reftable_record copy = reftable_record_for(rec->type);\n+\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \t/* do it twice to catch memory leaks */\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n-\tswitch (reftable_record_type(©)) {\n+\tswitch (copy.type) {\n \tcase BLOCK_TYPE_REF:\n-\t\tEXPECT(reftable_ref_record_equal(reftable_record_as_ref(©),\n-\t\t\t\t\t\t reftable_record_as_ref(rec),\n+\t\tEXPECT(reftable_ref_record_equal(©.u.ref, &rec->u.ref,\n \t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\tbreak;\n \tcase BLOCK_TYPE_LOG:\n-\t\tEXPECT(reftable_log_record_equal(reftable_record_as_log(©),\n-\t\t\t\t\t\t reftable_record_as_log(rec),\n+\t\tEXPECT(reftable_log_record_equal(©.u.log, &rec->u.log,\n \t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\tbreak;\n \t}\n-\treftable_record_destroy(©);\n+\treftable_record_release(©);\n }\n \n static void test_varint_roundtrip(void)\n@@ -106,61 +104,60 @@ static void test_reftable_ref_record_roundtrip(void)\n \tint i = 0;\n \n \tfor (i = REFTABLE_REF_DELETION; i < REFTABLE_NR_REF_VALUETYPES; i++) {\n-\t\tstruct reftable_ref_record in = { NULL };\n-\t\tstruct reftable_ref_record out = { NULL };\n-\t\tstruct reftable_record rec_out = { NULL };\n+\t\tstruct reftable_record in = {\n+\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t};\n+\t\tstruct reftable_record out = {\n+\t\t\t.type = BLOCK_TYPE_REF\n+\t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n-\t\tstruct reftable_record rec = { NULL };\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n \t\t\t.buf = buffer,\n \t\t\t.len = sizeof(buffer),\n \t\t};\n-\n \t\tint n, m;\n \n-\t\tin.value_type = i;\n+\t\tin.u.ref.value_type = i;\n \t\tswitch (i) {\n \t\tcase REFTABLE_REF_DELETION:\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_VAL1:\n-\t\t\tin.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val1, 1);\n+\t\t\tin.u.ref.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n+\t\t\tset_hash(in.u.ref.value.val1, 1);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_VAL2:\n-\t\t\tin.value.val2.value = reftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val2.value, 1);\n-\t\t\tin.value.val2.target_value =\n+\t\t\tin.u.ref.value.val2.value =\n+\t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n+\t\t\tset_hash(in.u.ref.value.val2.value, 1);\n+\t\t\tin.u.ref.value.val2.target_value =\n \t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val2.target_value, 2);\n+\t\t\tset_hash(in.u.ref.value.val2.target_value, 2);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_SYMREF:\n-\t\t\tin.value.symref = xstrdup(\"target\");\n+\t\t\tin.u.ref.value.symref = xstrdup(\"target\");\n \t\t\tbreak;\n \t\t}\n-\t\tin.refname = xstrdup(\"refs/heads/master\");\n+\t\tin.u.ref.refname = xstrdup(\"refs/heads/master\");\n \n-\t\treftable_record_from_ref(&rec, &in);\n-\t\ttest_copy(&rec);\n+\t\ttest_copy(&in);\n \n-\t\tEXPECT(reftable_record_val_type(&rec) == i);\n+\t\tEXPECT(reftable_record_val_type(&in) == i);\n \n-\t\treftable_record_key(&rec, &key);\n-\t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\t\treftable_record_key(&in, &key);\n+\t\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n > 0);\n \n \t\t/* decode into a non-zero reftable_record to test for leaks. */\n-\n-\t\treftable_record_from_ref(&rec_out, &out);\n-\t\tm = reftable_record_decode(&rec_out, key, i, dest,\n-\t\t\t\t\t GIT_SHA1_RAWSZ);\n+\t\tm = reftable_record_decode(&out, key, i, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_ref_record_equal(&in, &out, GIT_SHA1_RAWSZ));\n-\t\treftable_record_release(&rec_out);\n+\t\tEXPECT(reftable_ref_record_equal(&in.u.ref, &out.u.ref,\n+\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n+\t\treftable_record_release(&in);\n \n \t\tstrbuf_release(&key);\n-\t\treftable_ref_record_release(&in);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n@@ -213,7 +210,9 @@ static void test_reftable_log_record_roundtrip(void)\n \tset_test_hash(in[0].value.update.new_hash, 1);\n \tset_test_hash(in[0].value.update.old_hash, 2);\n \tfor (i = 0; i < ARRAY_SIZE(in); i++) {\n-\t\tstruct reftable_record rec = { NULL };\n+\t\tstruct reftable_record rec = {\n+\t\t\t.type = BLOCK_TYPE_LOG\n+\t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n@@ -221,23 +220,25 @@ static void test_reftable_log_record_roundtrip(void)\n \t\t\t.len = sizeof(buffer),\n \t\t};\n \t\t/* populate out, to check for leaks. */\n-\t\tstruct reftable_log_record out = {\n-\t\t\t.refname = xstrdup(\"old name\"),\n-\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n-\t\t\t.value = {\n-\t\t\t\t.update = {\n-\t\t\t\t\t.new_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n-\t\t\t\t\t.old_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n-\t\t\t\t\t.name = xstrdup(\"old name\"),\n-\t\t\t\t\t.email = xstrdup(\"old@email\"),\n-\t\t\t\t\t.message = xstrdup(\"old message\"),\n+\t\tstruct reftable_record out = {\n+\t\t\t.type = BLOCK_TYPE_LOG,\n+\t\t\t.u.log = {\n+\t\t\t\t.refname = xstrdup(\"old name\"),\n+\t\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n+\t\t\t\t.value = {\n+\t\t\t\t\t.update = {\n+\t\t\t\t\t\t.new_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t\t.old_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t\t.name = xstrdup(\"old name\"),\n+\t\t\t\t\t\t.email = xstrdup(\"old@email\"),\n+\t\t\t\t\t\t.message = xstrdup(\"old message\"),\n+\t\t\t\t\t},\n \t\t\t\t},\n \t\t\t},\n \t\t};\n-\t\tstruct reftable_record rec_out = { NULL };\n \t\tint n, m, valtype;\n \n-\t\treftable_record_from_log(&rec, &in[i]);\n+\t\trec.u.log = in[i];\n \n \t\ttest_copy(&rec);\n \n@@ -245,16 +246,16 @@ static void test_reftable_log_record_roundtrip(void)\n \n \t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n >= 0);\n-\t\treftable_record_from_log(&rec_out, &out);\n \t\tvaltype = reftable_record_val_type(&rec);\n-\t\tm = reftable_record_decode(&rec_out, key, valtype, dest,\n+\t\tm = reftable_record_decode(&out, key, valtype, dest,\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_log_record_equal(&in[i], &out, GIT_SHA1_RAWSZ));\n+\t\tEXPECT(reftable_log_record_equal(&in[i], &out.u.log,\n+\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\treftable_log_record_release(&in[i]);\n \t\tstrbuf_release(&key);\n-\t\treftable_record_release(&rec_out);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n@@ -322,47 +323,52 @@ static void test_reftable_obj_record_roundtrip(void)\n \t\t\t\t\t } };\n \tint i = 0;\n \tfor (i = 0; i < ARRAY_SIZE(recs); i++) {\n-\t\tstruct reftable_obj_record in = recs[i];\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n \t\t\t.buf = buffer,\n \t\t\t.len = sizeof(buffer),\n \t\t};\n-\t\tstruct reftable_record rec = { NULL };\n+\t\tstruct reftable_record in = {\n+\t\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t\t.u.obj = recs[i],\n+\t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n-\t\tstruct reftable_obj_record out = { NULL };\n-\t\tstruct reftable_record rec_out = { NULL };\n+\t\tstruct reftable_record out = {\n+\t\t\t.type = BLOCK_TYPE_OBJ\n+\t\t};\n \t\tint n, m;\n \t\tuint8_t extra;\n \n-\t\treftable_record_from_obj(&rec, &in);\n-\t\ttest_copy(&rec);\n-\t\treftable_record_key(&rec, &key);\n-\t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\t\ttest_copy(&in);\n+\t\treftable_record_key(&in, &key);\n+\t\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n > 0);\n-\t\textra = reftable_record_val_type(&rec);\n-\t\treftable_record_from_obj(&rec_out, &out);\n-\t\tm = reftable_record_decode(&rec_out, key, extra, dest,\n+\t\textra = reftable_record_val_type(&in);\n+\t\tm = reftable_record_decode(&out, key, extra, dest,\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(in.hash_prefix_len == out.hash_prefix_len);\n-\t\tEXPECT(in.offset_len == out.offset_len);\n-\n-\t\tEXPECT(!memcmp(in.hash_prefix, out.hash_prefix,\n-\t\t\t in.hash_prefix_len));\n-\t\tEXPECT(0 == memcmp(in.offsets, out.offsets,\n-\t\t\t\t sizeof(uint64_t) * in.offset_len));\n+\t\tEXPECT(in.u.obj.hash_prefix_len == out.u.obj.hash_prefix_len);\n+\t\tEXPECT(in.u.obj.offset_len == out.u.obj.offset_len);\n+\t\tif (in.u.obj.hash_prefix_len)\n+\t\t\tEXPECT(!memcmp(in.u.obj.hash_prefix, out.u.obj.hash_prefix,\n+\t\t\t\t in.u.obj.hash_prefix_len));\n+\t\tif (in.u.obj.offset_len)\n+\t\t\tEXPECT(!memcmp(in.u.obj.offsets, out.u.obj.offsets,\n+\t\t\t\t\t sizeof(uint64_t) * in.u.obj.offset_len));\n \t\tstrbuf_release(&key);\n-\t\treftable_record_release(&rec_out);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n static void test_reftable_index_record_roundtrip(void)\n {\n-\tstruct reftable_index_record in = {\n-\t\t.offset = 42,\n-\t\t.last_key = STRBUF_INIT,\n+\tstruct reftable_record in = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = {\n+\t\t\t.offset = 42,\n+\t\t\t.last_key = STRBUF_INIT,\n+\t\t},\n \t};\n \tuint8_t buffer[1024] = { 0 };\n \tstruct string_view dest = {\n@@ -370,31 +376,30 @@ static void test_reftable_index_record_roundtrip(void)\n \t\t.len = sizeof(buffer),\n \t};\n \tstruct strbuf key = STRBUF_INIT;\n-\tstruct reftable_record rec = { NULL };\n-\tstruct reftable_index_record out = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record out_rec = { NULL };\n+\tstruct reftable_record out = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = { .last_key = STRBUF_INIT },\n+\t};\n \tint n, m;\n \tuint8_t extra;\n \n-\tstrbuf_addstr(&in.last_key, \"refs/heads/master\");\n-\treftable_record_from_index(&rec, &in);\n-\treftable_record_key(&rec, &key);\n-\ttest_copy(&rec);\n+\tstrbuf_addstr(&in.u.idx.last_key, \"refs/heads/master\");\n+\treftable_record_key(&in, &key);\n+\ttest_copy(&in);\n \n-\tEXPECT(0 == strbuf_cmp(&key, &in.last_key));\n-\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\tEXPECT(0 == strbuf_cmp(&key, &in.u.idx.last_key));\n+\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(n > 0);\n \n-\textra = reftable_record_val_type(&rec);\n-\treftable_record_from_index(&out_rec, &out);\n-\tm = reftable_record_decode(&out_rec, key, extra, dest, GIT_SHA1_RAWSZ);\n+\textra = reftable_record_val_type(&in);\n+\tm = reftable_record_decode(&out, key, extra, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(m == n);\n \n-\tEXPECT(in.offset == out.offset);\n+\tEXPECT(in.u.idx.offset == out.u.idx.offset);\n \n-\treftable_record_release(&out_rec);\n+\treftable_record_release(&out);\n \tstrbuf_release(&key);\n-\tstrbuf_release(&in.last_key);\n+\tstrbuf_release(&in.u.idx.last_key);\n }\n \n int record_test_main(int argc, const char *argv[])\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex 3ca721e9f64..f9544c5f97b 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -221,10 +221,10 @@ static int writer_add_record(struct reftable_writer *w,\n \tstrbuf_reset(&w->last_key);\n \tstrbuf_addbuf(&w->last_key, &key);\n \tif (w->block_writer == NULL) {\n-\t\twriter_reinit_block_writer(w, reftable_record_type(rec));\n+\t\twriter_reinit_block_writer(w, rec->type);\n \t}\n \n-\tassert(block_writer_type(w->block_writer) == reftable_record_type(rec));\n+\tassert(block_writer_type(w->block_writer) == rec->type);\n \n \tif (block_writer_add(w->block_writer, rec) == 0) {\n \t\terr = 0;\n@@ -236,7 +236,7 @@ static int writer_add_record(struct reftable_writer *w,\n \t\tgoto done;\n \t}\n \n-\twriter_reinit_block_writer(w, reftable_record_type(rec));\n+\twriter_reinit_block_writer(w, rec->type);\n \terr = block_writer_add(w->block_writer, rec);\n \tif (err < 0) {\n \t\tgoto done;\n@@ -251,8 +251,10 @@ done:\n int reftable_writer_add_ref(struct reftable_writer *w,\n \t\t\t struct reftable_ref_record *ref)\n {\n-\tstruct reftable_record rec = { NULL };\n-\tstruct reftable_ref_record copy = *ref;\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = *ref,\n+\t};\n \tint err = 0;\n \n \tif (ref->refname == NULL)\n@@ -261,8 +263,7 @@ int reftable_writer_add_ref(struct reftable_writer *w,\n \t ref->update_index > w->max_update_index)\n \t\treturn REFTABLE_API_ERROR;\n \n-\treftable_record_from_ref(&rec, ©);\n-\tcopy.update_index -= w->min_update_index;\n+\trec.u.ref.update_index -= w->min_update_index;\n \n \terr = writer_add_record(w, &rec);\n \tif (err < 0)\n@@ -301,7 +302,10 @@ int reftable_writer_add_refs(struct reftable_writer *w,\n static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n \t\t\t\t\t struct reftable_log_record *log)\n {\n-\tstruct reftable_record rec = { NULL };\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.u.log = *log,\n+\t};\n \tif (w->block_writer &&\n \t block_writer_type(w->block_writer) == BLOCK_TYPE_REF) {\n \t\tint err = writer_finish_public_section(w);\n@@ -311,8 +315,6 @@ static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n \n \tw->next -= w->pending_padding;\n \tw->pending_padding = 0;\n-\n-\treftable_record_from_log(&rec, log);\n \treturn writer_add_record(w, &rec);\n }\n \n@@ -393,8 +395,10 @@ static int writer_finish_section(struct reftable_writer *w)\n \t\tw->index_len = 0;\n \t\tw->index_cap = 0;\n \t\tfor (i = 0; i < idx_len; i++) {\n-\t\t\tstruct reftable_record rec = { NULL };\n-\t\t\treftable_record_from_index(&rec, idx + i);\n+\t\t\tstruct reftable_record rec = {\n+\t\t\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t\t\t.u.idx = idx[i],\n+\t\t\t};\n \t\t\tif (block_writer_add(w->block_writer, &rec) == 0) {\n \t\t\t\tcontinue;\n \t\t\t}\n@@ -462,17 +466,18 @@ static void write_object_record(void *void_arg, void *key)\n {\n \tstruct write_record_arg *arg = void_arg;\n \tstruct obj_index_tree_node *entry = key;\n-\tstruct reftable_obj_record obj_rec = {\n-\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n-\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n-\t\t.offsets = entry->offsets,\n-\t\t.offset_len = entry->offset_len,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.u.obj = {\n+\t\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n+\t\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n+\t\t\t.offsets = entry->offsets,\n+\t\t\t.offset_len = entry->offset_len,\n+\t\t}\n \t};\n-\tstruct reftable_record rec = { NULL };\n \tif (arg->err < 0)\n \t\tgoto done;\n \n-\treftable_record_from_obj(&rec, &obj_rec);\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \tif (arg->err == 0)\n \t\tgoto done;\n@@ -485,7 +490,8 @@ static void write_object_record(void *void_arg, void *key)\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \tif (arg->err == 0)\n \t\tgoto done;\n-\tobj_rec.offset_len = 0;\n+\n+\trec.u.obj.offset_len = 0;\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \n \t/* Should be able to write into a fresh block. */\n-- \ngitgitgadget\n"},{"id":"443605","messageId":"YbGUsY+wjDs0UgxQ@coredump.intra.peff.net","threadId":"57047","inReplyTo":"pull.1152.v2.git.git.1639000187.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 00/11] Reftable coverity fixes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-12-09T05:31:29Z","receivedAt":"2021-12-09T05:31:41Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Dec 08, 2021 at 09:49:35PM +0000, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> 2: 603bd1d4f6e ! 2: 1ddcfe61ebc reftable: fix resource leak in error path\n> @@ reftable/block.c: int block_reader_init(struct block_reader *br, struct reftable\n> \n> \tif (typ == BLOCK_TYPE_LOG) {\n> \t\tint block_header_skip = 4 + header_off;\n> +@@ reftable/block.c: int block_reader_init(struct block_reader *br, struct reftable_block *block,\n> + \t\tuLongf src_len = block->len - block_header_skip;\n> + \t\t/* Log blocks specify the *uncompressed* size in their header.\n> + \t\t */\n> +-\t\tuint8_t *uncompressed = reftable_malloc(sz);\n> ++\t\tuncompressed = reftable_malloc(sz);\n> + \n> + \t\t/* Copy over the block header verbatim. It's not compressed. */\n> + \t\tmemcpy(uncompressed, block->data, block_header_skip);\n\nThis drops the shadowed variable, but it needs a little more. In one of\nthe error blocks we still free the variable, and then jump to the\n\"done\" label, which will try to free it again. So you need:\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 0bc4a5d84b..6d55817e0d 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -215,7 +215,6 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\tif (Z_OK !=\n \t\t uncompress2(uncompressed + block_header_skip, &dst_len,\n \t\t\t\tblock->data + block_header_skip, &src_len)) {\n-\t\t\treftable_free(uncompressed);\n \t\t\terr = REFTABLE_ZLIB_ERROR;\n \t\t\tgoto done;\n \t\t}\n\nto avoid the double-free. I think it's otherwise OK. The other error\ncase does not try to free, and in the success case we hand off ownership\nof \"uncompressed\" and set it back to NULL.\n\nI doubt this is the cause of the segfault on Windows, though (I saw it\nvia Coverity). That one still puzzles me, and I suspect will need more\noutput to figure out. It's possible to get an interactive session on the\nWindows VM running the CI:\n\n https://github.com/marketplace/actions/debugging-with-tmate\n\nbut I haven't tried it myself. I'm also not sure what tools are\navailable there (gdb sure would be nice, but even a backtrace of the\nsegfault would be quite a lot; I wonder if we could build with ASan on\nWindows).\n\n-Peff\n"},{"id":"443645","messageId":"211209.867dcekm9h.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"YbEMnksMEuAz3Nt0@coredump.intra.peff.net","subject":"Re: [PATCH] config.mak.dev: specify -std=gnu99 for gcc/clang","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-12-09T12:05:44Z","receivedAt":"2021-12-09T12:15:11Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Dec 08 2021, Jeff King wrote:\n\n> On Tue, Dec 07, 2021 at 08:13:43PM -0800, Junio C Hamano wrote:\n>\n>> Jeff King writes:\n>> \n>> >> error: ISO C99 doesn't support unnamed structs/unions [-Werror=pedantic]\n>> >\n>> > Hmm. It's interesting that the regular DEVELOPER=1 doesn't catch this.\n>> > It's because we don't specify -std there, and newer gcc defaults to\n>> > gnu17 (unnamed unions appeared in c11, I think). I wonder if it would be\n>> > helpful to teach config.mak.dev to pass -std=c99.\n>> \n>> FWIW, I use -std=gnu99 as our Makefile suggests.\n>\n> I think the patch below would have detected this both locally for\n> Han-Wen, but also in C.\n>\n> -- >8 --\n> Subject: [PATCH] config.mak.dev: specify -std=gnu99 for gcc/clang\n>\n> The point of DEVELOPER=1 is to turn up the warnings so we can catch\n> portability or correctness mistakes at the compiler level. But since\n> modern compilers tend to default to modern standards like gnu17, we\n> might miss warnings about older standards, even though we expect Git to\n> build with compilers that use them.\n>\n> So it's helpful for developer builds to set the -std argument to our\n> lowest-common denominator. Traditionally this was c89, but since we're\n> moving to assuming c99 in 7bc341e21b (git-compat-util: add a test\n> balloon for C99 support, 2021-12-01) that seems like a good spot to\n> land. And as explained in that commit, we want \"gnu99\" because we still\n> want to take advantage of some extensions when they're available.\n>\n> The new argument kicks in only for clang and gcc (which we know to\n> support \"-std=\" and \"gnu\" standards). And only for compiler versions\n> which default to a newer standard. That will avoid accidentally\n> silencing any build problems that non-developers would run into on older\n> compilers that default to c89.\n>\n> My digging found that the default switched to gnu11 in gcc 5.1.0.\n> Clang's documentation is less clear, but has done so since at least\n> clang-7. So that's what I put in the conditional here. It's OK to err on\n> the side of not-enabling this for older compilers. Most developers (as\n> well as CI) are using much more recent versions, so any warnings will\n> eventually surface.\n>\n> A concrete example is anonymous unions, which became legal in c11.\n> Without this patch, \"gcc -pedantic\" will not complain about them, but\n> will if we add in \"-std=gnu99\".\n>\n> Signed-off-by: Jeff King \n> ---\n> config.mak.dev | 5 +++++\n> 1 file changed, 5 insertions(+)\n>\n> diff --git a/config.mak.dev b/config.mak.dev\n> index 7673fed114..d4afac6b51 100644\n> --- a/config.mak.dev\n> +++ b/config.mak.dev\n> @@ -19,6 +19,11 @@ endif\n> endif\n> endif\n> endif\n> +\n> +ifneq ($(or $(filter gcc6,$(COMPILER_FEATURES)),$(filter clang7,$(COMPILER_FEATURES))),)\n> +DEVELOPER_CFLAGS += -std=gnu99\n> +endif\n> +\n> DEVELOPER_CFLAGS += -Wdeclaration-after-statement\n> DEVELOPER_CFLAGS += -Wformat-security\n> DEVELOPER_CFLAGS += -Wold-style-definition\n\nThis approach looks good & the rationale make sense.\n\nI mentioned in [1] that this might be a bad idea because:\n\n And as you note it's not only that older or non-gcc non-clang compilers\n may not understand this at all, but are we getting worse behavior on\n modern versions of those two because they're forced into some 20 year\n old C99 standard mode, instead of the current preferred default?\n\nBut from some short testing of GCC it will generate the exact same\n.s regardless of -std=* option, so I think this indeed only\nimpacts the warnings we'll emit. So pinning this seems to categorically\nbe a good thing.\n\nA bad thing about this is that we'll explicitly avoid happy accidents\nwhere we start relying on some newer C standard, and discover N releases\nlater that it was no big deal, and can thus just use that feature.\n\nOn the other hand having this means less back & forth churn of adding\nsuch a dependency only to find it breaks on one of our platforms\netc. Overall I think this makes sense, just say'n.\n\nI don't think this needs to change, but FWIW this would benefit from the\nsame sort of \"let's just compile it\" step as [2]. I.e. I think you'll\nfind that we could just check the exit code of:\n\n $(CC) -E -std=gnu99 - > +\n> > +ifneq ($(or $(filter gcc6,$(COMPILER_FEATURES)),$(filter clang7,$(COMPILER_FEATURES))),)\n> > +DEVELOPER_CFLAGS += -std=gnu99\n> > +endif\n> [...]\n> \n> This approach looks good & the rationale make sense.\n> \n> I mentioned in [1] that this might be a bad idea because:\n> \n> And as you note it's not only that older or non-gcc non-clang compilers\n> may not understand this at all, but are we getting worse behavior on\n> modern versions of those two because they're forced into some 20 year\n> old C99 standard mode, instead of the current preferred default?\n> \n> But from some short testing of GCC it will generate the exact same\n> .s regardless of -std=* option, so I think this indeed only\n> impacts the warnings we'll emit. So pinning this seems to categorically\n> be a good thing.\n\nThanks for looking into that. The thought crossed my mind, too, but I\ndidn't have any actual data. I think some of this is due to the standard\ncommittee keeping backwards compatibility. I.e., it's unlikely for there\nto be a case where the N standard says \"you must do X\", and N+1 says\n\"that's dumb, you can do Y instead\". Usually it is about new features\nthat were syntactically invalid or created undefined behavior in version\nN.\n\n> A bad thing about this is that we'll explicitly avoid happy accidents\n> where we start relying on some newer C standard, and discover N releases\n> later that it was no big deal, and can thus just use that feature.\n> \n> On the other hand having this means less back & forth churn of adding\n> such a dependency only to find it breaks on one of our platforms\n> etc. Overall I think this makes sense, just say'n.\n\nRight. I think it forces us to be more deliberate, but that's probably\nOK.\n\n> I don't think this needs to change, but FWIW this would benefit from the\n> same sort of \"let's just compile it\" step as [2]. I.e. I think you'll\n> find that we could just check the exit code of:\n> \n> $(CC) -E -std=gnu99 - \n> This works on GCC/Clang, and will die on xlc/suncc, and I assume any\n> other compiler that doesn't grok this. But I think it's better to avoid\n> a $(shell) here just for that, and any such change can wait until we\n> have some proper \"compile this once and cache it\" probing for\n> config.mak.dev.\n\nYou'd lose one property that the version-check has, which is that we\ndon't ever _upgrade_ to gnu99 from gnu89. I'm not sure how important\nthat is, though. It would have been bad if we took something like\nbrian's c99 patch, and developers and CI both failed to realize that it\nwas breaking the out-of-the-box build for older compilers (because\nbehind the scenes they were getting -std=gnu99 that non-developers do\nnot). But now that we've decided on that direction anyway, I'm not sure\nthere's anything left to be learned.\n\n-Peff\n"},{"id":"443987","messageId":"pull.1152.v3.git.git.1639411309.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v2.git.git.1639000187.gitgitgadget@gmail.com","subject":"[PATCH v3 00/11] Reftable coverity fixes","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-13T16:01:38Z","receivedAt":"2021-12-13T16:01:54Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"This series was targeted to 'next'.\n\nThis addresses some complaints from Coverity that Peff reported.\n\nv3:\n\n * revert some changes suggested by stolee\n * add unittest for zlib corruption.\n\nHan-Wen Nienhuys (11):\n reftable: fix OOB stack write in print functions\n reftable: fix resource leak in error path\n reftable: fix resource leak blocksource.c\n reftable: check reftable_stack_auto_compact() return value\n reftable: ignore remove() return value in stack_test.c\n reftable: fix resource warning\n reftable: fix NULL derefs in error paths\n reftable: order unittests by complexity\n reftable: drop stray printf in readwrite_test\n reftable: handle null refnames in reftable_ref_record_equal\n reftable: make reftable_record a tagged union\n\n reftable/block.c | 30 ++++---\n reftable/block_test.c | 22 ++---\n reftable/blocksource.c | 6 +-\n reftable/generic.c | 35 ++++----\n reftable/iter.c | 4 +-\n reftable/merged.c | 33 +++----\n reftable/pq.c | 3 +-\n reftable/pq_test.c | 31 +++----\n reftable/reader.c | 96 ++++++++++----------\n reftable/readwrite_test.c | 68 +++++++++++++-\n reftable/record.c | 185 ++++++++++++++++----------------------\n reftable/record.h | 42 ++++-----\n reftable/record_test.c | 181 +++++++++++++++++++------------------\n reftable/stack.c | 15 ++--\n reftable/stack_test.c | 3 +-\n reftable/writer.c | 40 +++++----\n t/helper/test-reftable.c | 9 +-\n 17 files changed, 431 insertions(+), 372 deletions(-)\n\n\nbase-commit: fae76fe5da3df25d752f2251b7ccda3f62813aa9\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1152%2Fhanwen%2Freftable-coverity-v3\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1152/hanwen/reftable-coverity-v3\nPull-Request: https://github.com/git/git/pull/1152\n\nRange-diff vs v2:\n\n 1: 7c033815183 = 1: 1c1a3ff92bd reftable: fix OOB stack write in print functions\n 2: 1ddcfe61ebc ! 2: 975a570d388 reftable: fix resource leak in error path\n @@ Metadata\n ## Commit message ##\n reftable: fix resource leak in error path\n \n - This would be triggered by corrupt files, so it doesn't have test coverage. This\n - was discovered by a Coverity scan.\n + Add test coverage for corrupt zlib data.\n +\n + This problem was discovered by a Coverity scan.\n \n Signed-off-by: Han-Wen Nienhuys \n \n @@ reftable/block.c: int block_reader_init(struct block_reader *br, struct reftable\n \t\t/* Copy over the block header verbatim. It's not compressed. */\n \t\tmemcpy(uncompressed, block->data, block_header_skip);\n @@ reftable/block.c: int block_reader_init(struct block_reader *br, struct reftable_block *block,\n + \t\tif (Z_OK !=\n \t\t uncompress2(uncompressed + block_header_skip, &dst_len,\n \t\t\t\tblock->data + block_header_skip, &src_len)) {\n - \t\t\treftable_free(uncompressed);\n +-\t\t\treftable_free(uncompressed);\n -\t\t\treturn REFTABLE_ZLIB_ERROR;\n +\t\t\terr = REFTABLE_ZLIB_ERROR;\n +\t\t\tgoto done;\n @@ reftable/block.c: int block_reader_init(struct block_reader *br, struct reftable\n }\n \n static uint32_t block_reader_restart_offset(struct block_reader *br, int i)\n +\n + ## reftable/readwrite_test.c ##\n +@@ reftable/readwrite_test.c: static void test_log_write_read(void)\n + \treader_close(&rd);\n + }\n + \n ++static void test_log_zlib_corruption(void)\n ++{\n ++\tstruct reftable_write_options opts = {\n ++\t\t.block_size = 256,\n ++\t};\n ++\tstruct reftable_iterator it = { NULL };\n ++\tstruct reftable_reader rd = { NULL };\n ++\tstruct reftable_block_source source = { NULL };\n ++\tstruct strbuf buf = STRBUF_INIT;\n ++\tstruct reftable_writer *w =\n ++\t\treftable_new_writer(&strbuf_add_void, &buf, &opts);\n ++\tconst struct reftable_stats *stats = NULL;\n ++\tuint8_t hash1[GIT_SHA1_RAWSZ] = { 1 };\n ++\tuint8_t hash2[GIT_SHA1_RAWSZ] = { 2 };\n ++\tchar message[100] = { 0 };\n ++\tint err, i, n;\n ++\n ++\tstruct reftable_log_record log = {\n ++\t\t.refname = \"refname\",\n ++\t\t.value_type = REFTABLE_LOG_UPDATE,\n ++\t\t.value = {\n ++\t\t\t.update = {\n ++\t\t\t\t.new_hash = hash1,\n ++\t\t\t\t.old_hash = hash2,\n ++\t\t\t\t.name = \"My Name\",\n ++\t\t\t\t.email = \"myname@invalid\",\n ++\t\t\t\t.message = message,\n ++\t\t\t},\n ++\t\t},\n ++\t};\n ++\n ++\tfor (i = 0; i < sizeof(message)-1; i++) {\n ++\t\tmessage[i] = (uint8_t)(rand() % 64 + ' ');\n ++\t}\n ++\n ++\treftable_writer_set_limits(w, 1, 1);\n ++\n ++\terr = reftable_writer_add_log(w, &log);\n ++\tEXPECT_ERR(err);\n ++\n ++\tn = reftable_writer_close(w);\n ++\tEXPECT(n == 0);\n ++\n ++\tstats = writer_stats(w);\n ++\tEXPECT(stats->log_stats.blocks > 0);\n ++\treftable_writer_free(w);\n ++\tw = NULL;\n ++\n ++\t/* corrupt the data. */\n ++\tbuf.buf[50] ^= 0x99;\n ++\n ++\tblock_source_from_strbuf(&source, &buf);\n ++\n ++\terr = init_reader(&rd, &source, \"file.log\");\n ++\tEXPECT_ERR(err);\n ++\n ++\terr = reftable_reader_seek_log(&rd, &it, \"refname\");\n ++\tEXPECT(err == REFTABLE_ZLIB_ERROR);\n ++\n ++\treftable_iterator_destroy(&it);\n ++\n ++\t/* cleanup. */\n ++\tstrbuf_release(&buf);\n ++\treader_close(&rd);\n ++}\n ++\n + static void test_table_read_write_sequential(void)\n + {\n + \tchar **names;\n +@@ reftable/readwrite_test.c: static void test_corrupt_table(void)\n + \n + int readwrite_test_main(int argc, const char *argv[])\n + {\n ++\tRUN_TEST(test_log_zlib_corruption);\n + \tRUN_TEST(test_corrupt_table);\n + \tRUN_TEST(test_corrupt_table_empty);\n + \tRUN_TEST(test_log_write_read);\n 3: e052b2a61d6 = 3: 0b9c7176d71 reftable: fix resource leak blocksource.c\n 4: 9063137457b = 4: 1dda4ee717f reftable: check reftable_stack_auto_compact() return value\n 5: 5020be156ae = 5: 36858e2070b reftable: ignore remove() return value in stack_test.c\n 6: 64c18d01cad = 6: 80b1988b885 reftable: fix resource warning\n 7: 700387ac5d3 = 7: 2939286924c reftable: fix NULL derefs in error paths\n 8: 713f1d09f68 = 8: 9dce18d7349 reftable: order unittests by complexity\n 9: cb601b51a47 = 9: 6b0af68f0b9 reftable: drop stray printf in readwrite_test\n 10: a0f83eff19f = 10: bff85cb0809 reftable: handle null refnames in reftable_ref_record_equal\n 11: a2743033cfd ! 11: b3e592b9c27 reftable: make reftable_record a tagged union\n @@ Commit message\n vtable within the structure.\n \n The only snag is that reftable_index_record contain a strbuf, so it cannot be\n - zero-initialized. To address this, introduce reftable_record_for() to create a\n - fresh instance, given a record type.\n + zero-initialized. To address this, use reftable_new_record() to return fresh\n + instance, given a record type. Since reftable_new_record() doesn't cause heap\n + allocation anymore, it should be balanced with reftable_record_release() rather\n + than reftable_record_destroy().\n \n Thanks to Peff for the suggestion.\n \n @@ Commit message\n Signed-off-by: Han-Wen Nienhuys \n \n ## reftable/block.c ##\n -@@ reftable/block.c: int block_reader_seek(struct block_reader *br, struct block_iter *it,\n - \t\t.key = *want,\n - \t\t.r = br,\n - \t};\n --\tstruct reftable_record rec = reftable_new_record(block_reader_type(br));\n -+\tstruct reftable_record rec = reftable_record_for(block_reader_type(br));\n - \tstruct strbuf key = STRBUF_INIT;\n - \tint err = 0;\n - \tstruct block_iter next = {\n @@ reftable/block.c: int block_reader_seek(struct block_reader *br, struct block_iter *it,\n done:\n \tstrbuf_release(&key);\n @@ reftable/iter.c: static int indexed_table_ref_iter_next_block(struct indexed_tab\n \n ## reftable/merged.c ##\n @@ reftable/merged.c: static int merged_iter_init(struct merged_iter *mi)\n - {\n - \tint i = 0;\n - \tfor (i = 0; i < mi->stack_len; i++) {\n --\t\tstruct reftable_record rec = reftable_new_record(mi->typ);\n -+\t\tstruct reftable_record rec = reftable_record_for(mi->typ);\n - \t\tint err = iterator_next(&mi->stack[i], &rec);\n - \t\tif (err < 0) {\n - \t\t\treturn err;\n -@@ reftable/merged.c: static int merged_iter_init(struct merged_iter *mi)\n \n \t\tif (err > 0) {\n \t\t\treftable_iterator_destroy(&mi->stack[i]);\n @@ reftable/merged.c: static void merged_iter_close(void *p)\n -\tstruct reftable_record rec = reftable_new_record(mi->typ);\n \tstruct pq_entry e = {\n -\t\t.rec = rec,\n -+\t\t.rec = reftable_record_for(mi->typ),\n ++\t\t.rec = reftable_new_record(mi->typ),\n \t\t.index = idx,\n \t};\n -\tint err = iterator_next(&mi->stack[idx], &rec);\n @@ reftable/merged.c: static int merged_iter_next_entry(struct merged_iter *mi,\n \tstrbuf_release(&entry_key);\n \treturn 0;\n }\n -@@ reftable/merged.c: static int merged_table_seek_record(struct reftable_merged_table *mt,\n - \t\tsizeof(struct reftable_iterator) * mt->stack_len);\n - \tstruct merged_iter merged = {\n - \t\t.stack = iters,\n --\t\t.typ = reftable_record_type(rec),\n -+\t\t.typ = rec->type,\n - \t\t.hash_id = mt->hash_id,\n - \t\t.suppress_deletions = mt->suppress_deletions,\n - \t};\n @@ reftable/merged.c: int reftable_merged_table_seek_ref(struct reftable_merged_table *mt,\n \t\t\t\t struct reftable_iterator *it,\n \t\t\t\t const char *name)\n @@ reftable/pq_test.c: static void test_pq(void)\n +\t\tstruct reftable_record *rec = &e.rec;\n \t\tmerged_iter_pqueue_check(pq);\n \n -+\t\tEXPECT(rec->type == BLOCK_TYPE_REF);\n ++\t\tEXPECT(reftable_record_type(rec) == BLOCK_TYPE_REF);\n \t\tif (last) {\n -\t\t\tEXPECT(strcmp(last, ref->refname) < 0);\n +\t\t\tEXPECT(strcmp(last, rec->u.ref.refname) < 0);\n @@ reftable/pq_test.c: static void test_pq(void)\n \n ## reftable/reader.c ##\n @@ reftable/reader.c: static int table_iter_next_in_block(struct table_iter *ti,\n - \t\t\t\t struct reftable_record *rec)\n {\n \tint res = block_iter_next(&ti->bi, rec);\n --\tif (res == 0 && reftable_record_type(rec) == BLOCK_TYPE_REF) {\n + \tif (res == 0 && reftable_record_type(rec) == BLOCK_TYPE_REF) {\n -\t\t((struct reftable_ref_record *)rec->data)->update_index +=\n -\t\t\tti->r->min_update_index;\n -+\tif (res == 0 && rec->type == BLOCK_TYPE_REF) {\n +\t\trec->u.ref.update_index += ti->r->min_update_index;\n \t}\n \n \treturn res;\n -@@ reftable/reader.c: static int table_iter_next_block(struct table_iter *dest,\n - \n - static int table_iter_next(struct table_iter *ti, struct reftable_record *rec)\n - {\n --\tif (reftable_record_type(rec) != ti->typ)\n -+\tif (rec->type != ti->typ)\n - \t\treturn REFTABLE_API_ERROR;\n - \n - \twhile (1) {\n @@ reftable/reader.c: static int reader_start(struct reftable_reader *r, struct table_iter *ti,\n static int reader_seek_linear(struct reftable_reader *r, struct table_iter *ti,\n \t\t\t struct reftable_record *want)\n {\n -\tstruct reftable_record rec =\n -\t\treftable_new_record(reftable_record_type(want));\n -+\tstruct reftable_record rec = reftable_record_for(want->type);\n ++\tstruct reftable_record rec = reftable_new_record(reftable_record_type(want));\n \tstruct strbuf want_key = STRBUF_INIT;\n \tstruct strbuf got_key = STRBUF_INIT;\n \tstruct table_iter next = TABLE_ITER_INIT;\n @@ reftable/reader.c: static int reader_seek_indexed(struct reftable_reader *r,\n -\treftable_record_from_index(&want_index_rec, &want_index);\n -\treftable_record_from_index(&index_result_rec, &index_result);\n -\n --\terr = reader_start(r, &index_iter, reftable_record_type(rec), 1);\n +\treftable_record_key(rec, &want_index.u.idx.last_key);\n -+\terr = reader_start(r, &index_iter, rec->type, 1);\n + \terr = reader_start(r, &index_iter, reftable_record_type(rec), 1);\n \tif (err < 0)\n \t\tgoto done;\n \n @@ reftable/reader.c: static int reader_seek_indexed(struct reftable_reader *r,\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n --\t\tif (next.typ == reftable_record_type(rec)) {\n -+\t\tif (next.typ == rec->type) {\n - \t\t\terr = 0;\n - \t\t\tbreak;\n - \t\t}\n @@ reftable/reader.c: static int reader_seek_indexed(struct reftable_reader *r,\n done:\n \tblock_iter_close(&next.bi);\n @@ reftable/reader.c: static int reader_seek_internal(struct reftable_reader *r,\n {\n -\tstruct reftable_reader_offsets *offs =\n -\t\treader_offsets_for(r, reftable_record_type(rec));\n -+\tstruct reftable_reader_offsets *offs = reader_offsets_for(r, rec->type);\n ++\tstruct reftable_reader_offsets *offs = reader_offsets_for(r, reftable_record_type(rec));\n \tuint64_t idx = offs->index_offset;\n \tstruct table_iter ti = TABLE_ITER_INIT;\n \tint err = 0;\n - \tif (idx > 0)\n - \t\treturn reader_seek_indexed(r, it, rec);\n - \n --\terr = reader_start(r, &ti, reftable_record_type(rec), 0);\n -+\terr = reader_start(r, &ti, rec->type, 0);\n - \tif (err < 0)\n - \t\treturn err;\n - \terr = reader_seek_linear(r, &ti, rec);\n -@@ reftable/reader.c: static int reader_seek_internal(struct reftable_reader *r,\n - static int reader_seek(struct reftable_reader *r, struct reftable_iterator *it,\n - \t\t struct reftable_record *rec)\n - {\n --\tuint8_t typ = reftable_record_type(rec);\n -+\tuint8_t typ = rec->type;\n - \n - \tstruct reftable_reader_offsets *offs = reader_offsets_for(r, typ);\n - \tif (!offs->is_present) {\n @@ reftable/reader.c: static int reader_seek(struct reftable_reader *r, struct reftable_iterator *it,\n int reftable_reader_seek_ref(struct reftable_reader *r,\n \t\t\t struct reftable_iterator *it, const char *name)\n @@ reftable/record.c: static struct reftable_record_vtable reftable_index_record_vt\n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)\n {\n -\trec->ops->key(rec->data, dest);\n --}\n --\n --uint8_t reftable_record_type(struct reftable_record *rec)\n --{\n --\treturn rec->ops->type;\n +\treftable_record_vtable(rec)->key(reftable_record_data(rec), dest);\n }\n \n + uint8_t reftable_record_type(struct reftable_record *rec)\n + {\n +-\treturn rec->ops->type;\n ++\treturn rec->type;\n + }\n + \n int reftable_record_encode(struct reftable_record *rec, struct string_view dest,\n \t\t\t int hash_size)\n {\n @@ reftable/record.c: void string_view_consume(struct string_view *s, int n)\n +\tabort();\n +}\n +\n -+struct reftable_record reftable_record_for(uint8_t typ)\n ++struct reftable_record reftable_new_record(uint8_t typ)\n +{\n +\tstruct reftable_record clean_idx = {\n +\t\t.type = BLOCK_TYPE_INDEX,\n @@ reftable/record.h: struct reftable_record_vtable {\n int reftable_is_block_type(uint8_t typ);\n \n -/* creates a malloced record of the given type. Dispose with record_destroy */\n --struct reftable_record reftable_new_record(uint8_t typ);\n --\n ++/* return an initialized record for the given type */\n + struct reftable_record reftable_new_record(uint8_t typ);\n + \n /* Encode `key` into `dest`. Sets `is_restart` to indicate a restart. Returns\n - * number of bytes written. */\n - int reftable_encode_key(int *is_restart, struct string_view dest,\n @@ reftable/record.h: struct reftable_obj_record {\n \tint offset_len;\n };\n @@ reftable/record.h: struct reftable_obj_record {\n +/* record is a generic wrapper for different types of records. It is normally\n + * created on the stack, or embedded within another struct. If the type is\n + * known, a fresh instance can be initialized explicitly. Otherwise, use\n -+ * reftable_record_for() to initialize generically (as the index_record is not\n ++ * reftable_new_record() to initialize generically (as the index_record is not\n + * valid as 0-initialized structure)\n + */\n +struct reftable_record {\n @@ reftable/record.h: struct reftable_obj_record {\n +\n /* see struct record_vtable */\n \n -+/* return an initialized record for the given type */\n -+struct reftable_record reftable_record_for(uint8_t typ);\n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest);\n - uint8_t reftable_record_type(struct reftable_record *rec);\n - void reftable_record_copy_from(struct reftable_record *rec,\n @@ reftable/record.h: int reftable_record_decode(struct reftable_record *rec, struct strbuf key,\n \t\t\t int hash_size);\n int reftable_record_is_deletion(struct reftable_record *rec);\n @@ reftable/record_test.c\n {\n -\tstruct reftable_record copy =\n -\t\treftable_new_record(reftable_record_type(rec));\n -+\tstruct reftable_record copy = reftable_record_for(rec->type);\n ++\tstruct reftable_record copy = reftable_new_record(reftable_record_type(rec));\n +\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \t/* do it twice to catch memory leaks */\n @@ reftable/record_test.c: static void test_reftable_index_record_roundtrip(void)\n int record_test_main(int argc, const char *argv[])\n \n ## reftable/writer.c ##\n -@@ reftable/writer.c: static int writer_add_record(struct reftable_writer *w,\n - \tstrbuf_reset(&w->last_key);\n - \tstrbuf_addbuf(&w->last_key, &key);\n - \tif (w->block_writer == NULL) {\n --\t\twriter_reinit_block_writer(w, reftable_record_type(rec));\n -+\t\twriter_reinit_block_writer(w, rec->type);\n - \t}\n - \n --\tassert(block_writer_type(w->block_writer) == reftable_record_type(rec));\n -+\tassert(block_writer_type(w->block_writer) == rec->type);\n - \n - \tif (block_writer_add(w->block_writer, rec) == 0) {\n - \t\terr = 0;\n -@@ reftable/writer.c: static int writer_add_record(struct reftable_writer *w,\n - \t\tgoto done;\n - \t}\n - \n --\twriter_reinit_block_writer(w, reftable_record_type(rec));\n -+\twriter_reinit_block_writer(w, rec->type);\n - \terr = block_writer_add(w->block_writer, rec);\n - \tif (err < 0) {\n - \t\tgoto done;\n @@ reftable/writer.c: done:\n int reftable_writer_add_ref(struct reftable_writer *w,\n \t\t\t struct reftable_ref_record *ref)\n\n-- \ngitgitgadget\n"},{"id":"443988","messageId":"1c1a3ff92bd24f7c588ce480983cecb3508a0add.1639411309.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v3.git.git.1639411309.gitgitgadget@gmail.com","subject":"[PATCH v3 01/11] reftable: fix OOB stack write in print functions","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-13T16:01:39Z","receivedAt":"2021-12-13T16:01:57Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 6a5dac32dc6..8536bd03aa9 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -254,7 +254,7 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)\n void reftable_ref_record_print(struct reftable_ref_record *ref,\n \t\t\t uint32_t hash_id)\n {\n-\tchar hex[2 * GIT_SHA256_RAWSZ + 1] = { 0 }; /* BUG */\n+\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */\n \tprintf(\"ref{%s(%\" PRIu64 \") \", ref->refname, ref->update_index);\n \tswitch (ref->value_type) {\n \tcase REFTABLE_REF_SYMREF:\n@@ -586,7 +586,7 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {\n void reftable_log_record_print(struct reftable_log_record *log,\n \t\t\t uint32_t hash_id)\n {\n-\tchar hex[GIT_SHA256_RAWSZ + 1] = { 0 };\n+\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 };\n \n \tswitch (log->value_type) {\n \tcase REFTABLE_LOG_DELETION:\n-- \ngitgitgadget\n\n"},{"id":"443989","messageId":"975a570d388fca79546987f4683fcd33419aad98.1639411309.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v3.git.git.1639411309.gitgitgadget@gmail.com","subject":"[PATCH v3 02/11] reftable: fix resource leak in error path","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-13T16:01:40Z","receivedAt":"2021-12-13T16:01:59Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nAdd test coverage for corrupt zlib data.\n\nThis problem was discovered by a Coverity scan.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/block.c | 28 ++++++++++------\n reftable/readwrite_test.c | 67 +++++++++++++++++++++++++++++++++++++++\n 2 files changed, 86 insertions(+), 9 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 855e3f5c947..37a85b9576d 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -188,13 +188,16 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tuint32_t full_block_size = table_block_size;\n \tuint8_t typ = block->data[header_off];\n \tuint32_t sz = get_be24(block->data + header_off + 1);\n-\n+\tint err = 0;\n \tuint16_t restart_count = 0;\n \tuint32_t restart_start = 0;\n \tuint8_t *restart_bytes = NULL;\n+\tuint8_t *uncompressed = NULL;\n \n-\tif (!reftable_is_block_type(typ))\n-\t\treturn REFTABLE_FORMAT_ERROR;\n+\tif (!reftable_is_block_type(typ)) {\n+\t\terr = REFTABLE_FORMAT_ERROR;\n+\t\tgoto done;\n+\t}\n \n \tif (typ == BLOCK_TYPE_LOG) {\n \t\tint block_header_skip = 4 + header_off;\n@@ -203,7 +206,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\tuLongf src_len = block->len - block_header_skip;\n \t\t/* Log blocks specify the *uncompressed* size in their header.\n \t\t */\n-\t\tuint8_t *uncompressed = reftable_malloc(sz);\n+\t\tuncompressed = reftable_malloc(sz);\n \n \t\t/* Copy over the block header verbatim. It's not compressed. */\n \t\tmemcpy(uncompressed, block->data, block_header_skip);\n@@ -212,16 +215,19 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\tif (Z_OK !=\n \t\t uncompress2(uncompressed + block_header_skip, &dst_len,\n \t\t\t\tblock->data + block_header_skip, &src_len)) {\n-\t\t\treftable_free(uncompressed);\n-\t\t\treturn REFTABLE_ZLIB_ERROR;\n+\t\t\terr = REFTABLE_ZLIB_ERROR;\n+\t\t\tgoto done;\n \t\t}\n \n-\t\tif (dst_len + block_header_skip != sz)\n-\t\t\treturn REFTABLE_FORMAT_ERROR;\n+\t\tif (dst_len + block_header_skip != sz) {\n+\t\t\terr = REFTABLE_FORMAT_ERROR;\n+\t\t\tgoto done;\n+\t\t}\n \n \t\t/* We're done with the input data. */\n \t\treftable_block_done(block);\n \t\tblock->data = uncompressed;\n+\t\tuncompressed = NULL;\n \t\tblock->len = sz;\n \t\tblock->source = malloc_block_source();\n \t\tfull_block_size = src_len + block_header_skip;\n@@ -251,7 +257,11 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tbr->restart_count = restart_count;\n \tbr->restart_bytes = restart_bytes;\n \n-\treturn 0;\n+done:\n+\tif (uncompressed) {\n+\t\treftable_free(uncompressed);\n+\t}\n+\treturn err;\n }\n \n static uint32_t block_reader_restart_offset(struct block_reader *br, int i)\ndiff --git a/reftable/readwrite_test.c b/reftable/readwrite_test.c\nindex 5f6bcc2f775..42caf0bde4c 100644\n--- a/reftable/readwrite_test.c\n+++ b/reftable/readwrite_test.c\n@@ -254,6 +254,72 @@ static void test_log_write_read(void)\n \treader_close(&rd);\n }\n \n+static void test_log_zlib_corruption(void)\n+{\n+\tstruct reftable_write_options opts = {\n+\t\t.block_size = 256,\n+\t};\n+\tstruct reftable_iterator it = { NULL };\n+\tstruct reftable_reader rd = { NULL };\n+\tstruct reftable_block_source source = { NULL };\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tstruct reftable_writer *w =\n+\t\treftable_new_writer(&strbuf_add_void, &buf, &opts);\n+\tconst struct reftable_stats *stats = NULL;\n+\tuint8_t hash1[GIT_SHA1_RAWSZ] = { 1 };\n+\tuint8_t hash2[GIT_SHA1_RAWSZ] = { 2 };\n+\tchar message[100] = { 0 };\n+\tint err, i, n;\n+\n+\tstruct reftable_log_record log = {\n+\t\t.refname = \"refname\",\n+\t\t.value_type = REFTABLE_LOG_UPDATE,\n+\t\t.value = {\n+\t\t\t.update = {\n+\t\t\t\t.new_hash = hash1,\n+\t\t\t\t.old_hash = hash2,\n+\t\t\t\t.name = \"My Name\",\n+\t\t\t\t.email = \"myname@invalid\",\n+\t\t\t\t.message = message,\n+\t\t\t},\n+\t\t},\n+\t};\n+\n+\tfor (i = 0; i < sizeof(message)-1; i++) {\n+\t\tmessage[i] = (uint8_t)(rand() % 64 + ' ');\n+\t}\n+\n+\treftable_writer_set_limits(w, 1, 1);\n+\n+\terr = reftable_writer_add_log(w, &log);\n+\tEXPECT_ERR(err);\n+\n+\tn = reftable_writer_close(w);\n+\tEXPECT(n == 0);\n+\n+\tstats = writer_stats(w);\n+\tEXPECT(stats->log_stats.blocks > 0);\n+\treftable_writer_free(w);\n+\tw = NULL;\n+\n+\t/* corrupt the data. */\n+\tbuf.buf[50] ^= 0x99;\n+\n+\tblock_source_from_strbuf(&source, &buf);\n+\n+\terr = init_reader(&rd, &source, \"file.log\");\n+\tEXPECT_ERR(err);\n+\n+\terr = reftable_reader_seek_log(&rd, &it, \"refname\");\n+\tEXPECT(err == REFTABLE_ZLIB_ERROR);\n+\n+\treftable_iterator_destroy(&it);\n+\n+\t/* cleanup. */\n+\tstrbuf_release(&buf);\n+\treader_close(&rd);\n+}\n+\n static void test_table_read_write_sequential(void)\n {\n \tchar **names;\n@@ -633,6 +699,7 @@ static void test_corrupt_table(void)\n \n int readwrite_test_main(int argc, const char *argv[])\n {\n+\tRUN_TEST(test_log_zlib_corruption);\n \tRUN_TEST(test_corrupt_table);\n \tRUN_TEST(test_corrupt_table_empty);\n \tRUN_TEST(test_log_write_read);\n-- \ngitgitgadget\n\n"},{"id":"443990","messageId":"0b9c7176d7122253e20f0bc88fb1c79c3efb5dcf.1639411309.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v3.git.git.1639411309.gitgitgadget@gmail.com","subject":"[PATCH v3 03/11] reftable: fix resource leak blocksource.c","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-13T16:01:41Z","receivedAt":"2021-12-13T16:02:02Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis would be triggered in the unlikely event of fstat() failing on an opened\nfile.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/blocksource.c | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/reftable/blocksource.c b/reftable/blocksource.c\nindex 0044eecd9aa..2605371c28d 100644\n--- a/reftable/blocksource.c\n+++ b/reftable/blocksource.c\n@@ -134,8 +134,10 @@ int reftable_block_source_from_file(struct reftable_block_source *bs,\n \t}\n \n \terr = fstat(fd, &st);\n-\tif (err < 0)\n-\t\treturn -1;\n+\tif (err < 0) {\n+\t\tclose(fd);\n+\t\treturn REFTABLE_IO_ERROR;\n+\t}\n \n \tp = reftable_calloc(sizeof(struct file_block_source));\n \tp->size = st.st_size;\n-- \ngitgitgadget\n\n"},{"id":"443991","messageId":"36858e2070badcc057c53c630553ae731c0c7552.1639411309.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v3.git.git.1639411309.gitgitgadget@gmail.com","subject":"[PATCH v3 05/11] reftable: ignore remove() return value in stack_test.c","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-13T16:01:43Z","receivedAt":"2021-12-13T16:02:04Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nIf the cleanup fails, there is nothing we can do.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack_test.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/reftable/stack_test.c b/reftable/stack_test.c\nindex d628420e63a..4b7292945c3 100644\n--- a/reftable/stack_test.c\n+++ b/reftable/stack_test.c\n@@ -89,7 +89,7 @@ static void test_read_file(void)\n \t\tEXPECT(0 == strcmp(want[i], names[i]));\n \t}\n \tfree_names(names);\n-\tremove(fn);\n+\t(void) remove(fn);\n }\n \n static void test_parse_names(void)\n-- \ngitgitgadget\n\n"},{"id":"443992","messageId":"1dda4ee717fc6465e95d22863f84975f79265a91.1639411309.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v3.git.git.1639411309.gitgitgadget@gmail.com","subject":"[PATCH v3 04/11] reftable: check reftable_stack_auto_compact() return value","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-13T16:01:42Z","receivedAt":"2021-12-13T16:02:06Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nFixes a problem detected by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack_test.c | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/reftable/stack_test.c b/reftable/stack_test.c\nindex eb0b7228b0c..d628420e63a 100644\n--- a/reftable/stack_test.c\n+++ b/reftable/stack_test.c\n@@ -814,6 +814,7 @@ static void test_reftable_stack_auto_compaction(void)\n \t\tEXPECT_ERR(err);\n \n \t\terr = reftable_stack_auto_compact(st);\n+\t\tEXPECT_ERR(err);\n \t\tEXPECT(i < 3 || st->merged->stack_len < 2 * fastlog2(i));\n \t}\n \n-- \ngitgitgadget\n\n"},{"id":"443993","messageId":"80b1988b885cb52bfb2f5837b6fee4c333863096.1639411309.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v3.git.git.1639411309.gitgitgadget@gmail.com","subject":"[PATCH v3 06/11] reftable: fix resource warning","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-13T16:01:44Z","receivedAt":"2021-12-13T16:02:07Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis would trigger in the unlikely event that we are compacting, and the next\navailable file handle is 0.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack.c | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/reftable/stack.c b/reftable/stack.c\nindex df5021ebf08..10dfd370e8e 100644\n--- a/reftable/stack.c\n+++ b/reftable/stack.c\n@@ -877,7 +877,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \tstruct strbuf new_table_path = STRBUF_INIT;\n \tint err = 0;\n \tint have_lock = 0;\n-\tint lock_file_fd = 0;\n+\tint lock_file_fd = -1;\n \tint compact_count = last - first + 1;\n \tchar **listp = NULL;\n \tchar **delete_on_success =\n@@ -911,7 +911,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \t}\n \t/* Don't want to write to the lock for now. */\n \tclose(lock_file_fd);\n-\tlock_file_fd = 0;\n+\tlock_file_fd = -1;\n \n \thave_lock = 1;\n \terr = stack_uptodate(st);\n@@ -932,7 +932,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \n \t\tsublock_file_fd = open(subtab_lock.buf,\n \t\t\t\t O_EXCL | O_CREAT | O_WRONLY, 0644);\n-\t\tif (sublock_file_fd > 0) {\n+\t\tif (sublock_file_fd >= 0) {\n \t\t\tclose(sublock_file_fd);\n \t\t} else if (sublock_file_fd < 0) {\n \t\t\tif (errno == EEXIST) {\n@@ -1013,7 +1013,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \t\tgoto done;\n \t}\n \terr = close(lock_file_fd);\n-\tlock_file_fd = 0;\n+\tlock_file_fd = -1;\n \tif (err < 0) {\n \t\terr = REFTABLE_IO_ERROR;\n \t\tunlink(new_table_path.buf);\n@@ -1050,9 +1050,9 @@ done:\n \t\tlistp++;\n \t}\n \tfree_names(subtable_locks);\n-\tif (lock_file_fd > 0) {\n+\tif (lock_file_fd >= 0) {\n \t\tclose(lock_file_fd);\n-\t\tlock_file_fd = 0;\n+\t\tlock_file_fd = -1;\n \t}\n \tif (have_lock) {\n \t\tunlink(lock_file_name.buf);\n-- \ngitgitgadget\n\n"},{"id":"443994","messageId":"2939286924caa67e2cc57bf5ed4ce4287b9d571c.1639411309.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v3.git.git.1639411309.gitgitgadget@gmail.com","subject":"[PATCH v3 07/11] reftable: fix NULL derefs in error paths","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-13T16:01:45Z","receivedAt":"2021-12-13T16:02:07Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSpotted by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/reader.c | 3 ++-\n reftable/stack.c | 3 ++-\n 2 files changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 006709a645a..8d308d858f8 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -796,6 +796,7 @@ int reftable_reader_print_file(const char *tablename)\n \treftable_table_from_reader(&tab, r);\n \terr = reftable_table_print(&tab);\n done:\n-\treftable_reader_free(r);\n+\tif (r)\n+\t\treftable_reader_free(r);\n \treturn err;\n }\ndiff --git a/reftable/stack.c b/reftable/stack.c\nindex 10dfd370e8e..eb03b6c74f6 100644\n--- a/reftable/stack.c\n+++ b/reftable/stack.c\n@@ -707,7 +707,8 @@ done:\n \tstrbuf_release(&temp_tab_file_name);\n \tstrbuf_release(&tab_file_name);\n \tstrbuf_release(&next_name);\n-\treftable_writer_free(wr);\n+\tif (wr)\n+\t\treftable_writer_free(wr);\n \treturn err;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"443995","messageId":"9dce18d7349fe2a27859c224068fb5a386f263f1.1639411309.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v3.git.git.1639411309.gitgitgadget@gmail.com","subject":"[PATCH v3 08/11] reftable: order unittests by complexity","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-13T16:01:46Z","receivedAt":"2021-12-13T16:02:09Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis is a more pratical ordering when working on refactorings of the reftable\ncode.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n t/helper/test-reftable.c | 9 +++++----\n 1 file changed, 5 insertions(+), 4 deletions(-)\n\ndiff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c\nindex 26b03d7b789..f08d66df949 100644\n--- a/t/helper/test-reftable.c\n+++ b/t/helper/test-reftable.c\n@@ -3,15 +3,16 @@\n \n int cmd__reftable(int argc, const char **argv)\n {\n+\t// test from simple to complex.\n \tbasics_test_main(argc, argv);\n+\trecord_test_main(argc, argv);\n \tblock_test_main(argc, argv);\n-\tmerged_test_main(argc, argv);\n+\ttree_test_main(argc, argv);\n \tpq_test_main(argc, argv);\n-\trecord_test_main(argc, argv);\n-\trefname_test_main(argc, argv);\n \treadwrite_test_main(argc, argv);\n+\tmerged_test_main(argc, argv);\n \tstack_test_main(argc, argv);\n-\ttree_test_main(argc, argv);\n+\trefname_test_main(argc, argv);\n \treturn 0;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"443996","messageId":"bff85cb080975cb08e95e8df85afcc27da307cc6.1639411309.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v3.git.git.1639411309.gitgitgadget@gmail.com","subject":"[PATCH v3 10/11] reftable: handle null refnames in reftable_ref_record_equal","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-13T16:01:48Z","receivedAt":"2021-12-13T16:02:10Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSpotted by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 8536bd03aa9..8bbcbff1e69 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -1154,9 +1154,11 @@ int reftable_ref_record_equal(struct reftable_ref_record *a,\n \t\t\t struct reftable_ref_record *b, int hash_size)\n {\n \tassert(hash_size > 0);\n-\tif (!(0 == strcmp(a->refname, b->refname) &&\n-\t a->update_index == b->update_index &&\n-\t a->value_type == b->value_type))\n+\tif (!null_streq(a->refname, b->refname))\n+\t\treturn 0;\n+\n+\tif (a->update_index != b->update_index ||\n+\t a->value_type != b->value_type)\n \t\treturn 0;\n \n \tswitch (a->value_type) {\n-- \ngitgitgadget\n\n"},{"id":"443997","messageId":"6b0af68f0b94fb96c81e25f9a1911fa05dcf07fd.1639411309.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v3.git.git.1639411309.gitgitgadget@gmail.com","subject":"[PATCH v3 09/11] reftable: drop stray printf in readwrite_test","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-13T16:01:47Z","receivedAt":"2021-12-13T16:02:12Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/readwrite_test.c | 1 -\n 1 file changed, 1 deletion(-)\n\ndiff --git a/reftable/readwrite_test.c b/reftable/readwrite_test.c\nindex 42caf0bde4c..f6546d7b8d1 100644\n--- a/reftable/readwrite_test.c\n+++ b/reftable/readwrite_test.c\n@@ -663,7 +663,6 @@ static void test_write_key_order(void)\n \terr = reftable_writer_add_ref(w, &refs[0]);\n \tEXPECT_ERR(err);\n \terr = reftable_writer_add_ref(w, &refs[1]);\n-\tprintf(\"%d\\n\", err);\n \tEXPECT(err == REFTABLE_API_ERROR);\n \treftable_writer_close(w);\n \treftable_writer_free(w);\n-- \ngitgitgadget\n\n"},{"id":"443998","messageId":"b3e592b9c27a0c205b5f75e6f5df86d637f76f5d.1639411309.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v3.git.git.1639411309.gitgitgadget@gmail.com","subject":"[PATCH v3 11/11] reftable: make reftable_record a tagged union","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-13T16:01:49Z","receivedAt":"2021-12-13T16:02:13Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis reduces the amount of glue code, because we don't need a void pointer or\nvtable within the structure.\n\nThe only snag is that reftable_index_record contain a strbuf, so it cannot be\nzero-initialized. To address this, use reftable_new_record() to return fresh\ninstance, given a record type. Since reftable_new_record() doesn't cause heap\nallocation anymore, it should be balanced with reftable_record_release() rather\nthan reftable_record_destroy().\n\nThanks to Peff for the suggestion.\n\nHelped-by: Jeff King \nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/block.c | 2 +-\n reftable/block_test.c | 22 ++---\n reftable/generic.c | 35 ++++----\n reftable/iter.c | 4 +-\n reftable/merged.c | 33 ++++----\n reftable/pq.c | 3 +-\n reftable/pq_test.c | 31 +++----\n reftable/reader.c | 93 +++++++++++----------\n reftable/record.c | 173 ++++++++++++++++-----------------------\n reftable/record.h | 42 ++++------\n reftable/record_test.c | 181 +++++++++++++++++++++--------------------\n reftable/writer.c | 40 +++++----\n 12 files changed, 317 insertions(+), 342 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 37a85b9576d..872bedaa2f0 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -423,7 +423,7 @@ int block_reader_seek(struct block_reader *br, struct block_iter *it,\n done:\n \tstrbuf_release(&key);\n \tstrbuf_release(&next.last_key);\n-\treftable_record_destroy(&rec);\n+\treftable_record_release(&rec);\n \n \treturn err;\n }\ndiff --git a/reftable/block_test.c b/reftable/block_test.c\nindex 4b3ea262dcb..fa2ee092ec0 100644\n--- a/reftable/block_test.c\n+++ b/reftable/block_test.c\n@@ -26,8 +26,9 @@ static void test_block_read_write(void)\n \tstruct block_writer bw = {\n \t\t.last_key = STRBUF_INIT,\n \t};\n-\tstruct reftable_ref_record ref = { NULL };\n-\tstruct reftable_record rec = { NULL };\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t};\n \tint i = 0;\n \tint n;\n \tstruct block_reader br = { 0 };\n@@ -40,7 +41,6 @@ static void test_block_read_write(void)\n \tblock.source = malloc_block_source();\n \tblock_writer_init(&bw, BLOCK_TYPE_REF, block.data, block_size,\n \t\t\t header_off, hash_size(GIT_SHA1_FORMAT_ID));\n-\treftable_record_from_ref(&rec, &ref);\n \n \tfor (i = 0; i < N; i++) {\n \t\tchar name[100];\n@@ -48,14 +48,14 @@ static void test_block_read_write(void)\n \t\tsnprintf(name, sizeof(name), \"branch%02d\", i);\n \t\tmemset(hash, i, sizeof(hash));\n \n-\t\tref.refname = name;\n-\t\tref.value_type = REFTABLE_REF_VAL1;\n-\t\tref.value.val1 = hash;\n+\t\trec.u.ref.refname = name;\n+\t\trec.u.ref.value_type = REFTABLE_REF_VAL1;\n+\t\trec.u.ref.value.val1 = hash;\n \n \t\tnames[i] = xstrdup(name);\n \t\tn = block_writer_add(&bw, &rec);\n-\t\tref.refname = NULL;\n-\t\tref.value_type = REFTABLE_REF_DELETION;\n+\t\trec.u.ref.refname = NULL;\n+\t\trec.u.ref.value_type = REFTABLE_REF_DELETION;\n \t\tEXPECT(n == 0);\n \t}\n \n@@ -74,7 +74,7 @@ static void test_block_read_write(void)\n \t\tif (r > 0) {\n \t\t\tbreak;\n \t\t}\n-\t\tEXPECT_STREQ(names[j], ref.refname);\n+\t\tEXPECT_STREQ(names[j], rec.u.ref.refname);\n \t\tj++;\n \t}\n \n@@ -92,7 +92,7 @@ static void test_block_read_write(void)\n \t\tn = block_iter_next(&it, &rec);\n \t\tEXPECT(n == 0);\n \n-\t\tEXPECT_STREQ(names[i], ref.refname);\n+\t\tEXPECT_STREQ(names[i], rec.u.ref.refname);\n \n \t\twant.len--;\n \t\tn = block_reader_seek(&br, &it, &want);\n@@ -100,7 +100,7 @@ static void test_block_read_write(void)\n \n \t\tn = block_iter_next(&it, &rec);\n \t\tEXPECT(n == 0);\n-\t\tEXPECT_STREQ(names[10 * (i / 10)], ref.refname);\n+\t\tEXPECT_STREQ(names[10 * (i / 10)], rec.u.ref.refname);\n \n \t\tblock_iter_close(&it);\n \t}\ndiff --git a/reftable/generic.c b/reftable/generic.c\nindex 7a8a738d860..4446b8ed360 100644\n--- a/reftable/generic.c\n+++ b/reftable/generic.c\n@@ -7,6 +7,7 @@ https://developers.google.com/open-source/licenses/bsd\n */\n \n #include \"basics.h\"\n+#include \"constants.h\"\n #include \"record.h\"\n #include \"generic.h\"\n #include \"reftable-iterator.h\"\n@@ -15,23 +16,21 @@ https://developers.google.com/open-source/licenses/bsd\n int reftable_table_seek_ref(struct reftable_table *tab,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_REF,\n+\t\t\t\t .u.ref = {\n+\t\t\t\t\t .refname = (char *)name,\n+\t\t\t\t } };\n \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n }\n \n int reftable_table_seek_log(struct reftable_table *tab,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = ~((uint64_t)0),\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n+\t\t\t\t .u.log = {\n+\t\t\t\t\t .refname = (char *)name,\n+\t\t\t\t\t .update_index = ~((uint64_t)0),\n+\t\t\t\t } };\n \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n }\n \n@@ -129,17 +128,19 @@ void reftable_iterator_destroy(struct reftable_iterator *it)\n int reftable_iterator_next_ref(struct reftable_iterator *it,\n \t\t\t struct reftable_ref_record *ref)\n {\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, ref);\n-\treturn iterator_next(it, &rec);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_REF };\n+\tint err = iterator_next(it, &rec);\n+\t*ref = rec.u.ref;\n+\treturn err;\n }\n \n int reftable_iterator_next_log(struct reftable_iterator *it,\n \t\t\t struct reftable_log_record *log)\n {\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, log);\n-\treturn iterator_next(it, &rec);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG };\n+\tint err = iterator_next(it, &rec);\n+\t*log = rec.u.log;\n+\treturn err;\n }\n \n int iterator_next(struct reftable_iterator *it, struct reftable_record *rec)\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex 93d04f735b8..a8d174c0406 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -32,7 +32,7 @@ static int filtering_ref_iterator_next(void *iter_arg,\n \t\t\t\t struct reftable_record *rec)\n {\n \tstruct filtering_ref_iterator *fri = iter_arg;\n-\tstruct reftable_ref_record *ref = rec->data;\n+\tstruct reftable_ref_record *ref = &rec->u.ref;\n \tint err = 0;\n \twhile (1) {\n \t\terr = reftable_iterator_next_ref(&fri->it, ref);\n@@ -127,7 +127,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n static int indexed_table_ref_iter_next(void *p, struct reftable_record *rec)\n {\n \tstruct indexed_table_ref_iter *it = p;\n-\tstruct reftable_ref_record *ref = rec->data;\n+\tstruct reftable_ref_record *ref = &rec->u.ref;\n \n \twhile (1) {\n \t\tint err = block_iter_next(&it->cur, rec);\ndiff --git a/reftable/merged.c b/reftable/merged.c\nindex e5b53da6db3..a7c0df91ff0 100644\n--- a/reftable/merged.c\n+++ b/reftable/merged.c\n@@ -30,7 +30,7 @@ static int merged_iter_init(struct merged_iter *mi)\n \n \t\tif (err > 0) {\n \t\t\treftable_iterator_destroy(&mi->stack[i]);\n-\t\t\treftable_record_destroy(&rec);\n+\t\t\treftable_record_release(&rec);\n \t\t} else {\n \t\t\tstruct pq_entry e = {\n \t\t\t\t.rec = rec,\n@@ -57,18 +57,17 @@ static void merged_iter_close(void *p)\n static int merged_iter_advance_nonnull_subiter(struct merged_iter *mi,\n \t\t\t\t\t size_t idx)\n {\n-\tstruct reftable_record rec = reftable_new_record(mi->typ);\n \tstruct pq_entry e = {\n-\t\t.rec = rec,\n+\t\t.rec = reftable_new_record(mi->typ),\n \t\t.index = idx,\n \t};\n-\tint err = iterator_next(&mi->stack[idx], &rec);\n+\tint err = iterator_next(&mi->stack[idx], &e.rec);\n \tif (err < 0)\n \t\treturn err;\n \n \tif (err > 0) {\n \t\treftable_iterator_destroy(&mi->stack[idx]);\n-\t\treftable_record_destroy(&rec);\n+\t\treftable_record_release(&e.rec);\n \t\treturn 0;\n \t}\n \n@@ -126,11 +125,11 @@ static int merged_iter_next_entry(struct merged_iter *mi,\n \t\tif (err < 0) {\n \t\t\treturn err;\n \t\t}\n-\t\treftable_record_destroy(&top.rec);\n+\t\treftable_record_release(&top.rec);\n \t}\n \n \treftable_record_copy_from(rec, &entry.rec, hash_size(mi->hash_id));\n-\treftable_record_destroy(&entry.rec);\n+\treftable_record_release(&entry.rec);\n \tstrbuf_release(&entry_key);\n \treturn 0;\n }\n@@ -290,11 +289,12 @@ int reftable_merged_table_seek_ref(struct reftable_merged_table *mt,\n \t\t\t\t struct reftable_iterator *it,\n \t\t\t\t const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = {\n+\t\t\t.refname = (char *)name,\n+\t\t},\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n \treturn merged_table_seek_record(mt, it, &rec);\n }\n \n@@ -302,12 +302,13 @@ int reftable_merged_table_seek_log_at(struct reftable_merged_table *mt,\n \t\t\t\t struct reftable_iterator *it,\n \t\t\t\t const char *name, uint64_t update_index)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = update_index,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.u.log = {\n+\t\t\t.refname = (char *)name,\n+\t\t\t.update_index = update_index,\n+\t\t}\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n \treturn merged_table_seek_record(mt, it, &rec);\n }\n \ndiff --git a/reftable/pq.c b/reftable/pq.c\nindex efc474017a2..96ca6dd37b3 100644\n--- a/reftable/pq.c\n+++ b/reftable/pq.c\n@@ -74,6 +74,7 @@ struct pq_entry merged_iter_pqueue_remove(struct merged_iter_pqueue *pq)\n void merged_iter_pqueue_add(struct merged_iter_pqueue *pq, struct pq_entry e)\n {\n \tint i = 0;\n+\n \tif (pq->len == pq->cap) {\n \t\tpq->cap = 2 * pq->cap + 1;\n \t\tpq->heap = reftable_realloc(pq->heap,\n@@ -98,7 +99,7 @@ void merged_iter_pqueue_release(struct merged_iter_pqueue *pq)\n {\n \tint i = 0;\n \tfor (i = 0; i < pq->len; i++) {\n-\t\treftable_record_destroy(&pq->heap[i].rec);\n+\t\treftable_record_release(&pq->heap[i].rec);\n \t}\n \tFREE_AND_NULL(pq->heap);\n \tpq->len = pq->cap = 0;\ndiff --git a/reftable/pq_test.c b/reftable/pq_test.c\nindex c9bb05e37b7..e0f5c584ada 100644\n--- a/reftable/pq_test.c\n+++ b/reftable/pq_test.c\n@@ -31,7 +31,7 @@ static void test_pq(void)\n \tint N = ARRAY_SIZE(names) - 1;\n \n \tstruct merged_iter_pqueue pq = { NULL };\n-\tconst char *last = NULL;\n+\tchar *last = NULL;\n \n \tint i = 0;\n \tfor (i = 0; i < N; i++) {\n@@ -42,12 +42,14 @@ static void test_pq(void)\n \n \ti = 1;\n \tdo {\n-\t\tstruct reftable_record rec =\n-\t\t\treftable_new_record(BLOCK_TYPE_REF);\n-\t\tstruct pq_entry e = { 0 };\n-\n-\t\treftable_record_as_ref(&rec)->refname = names[i];\n-\t\te.rec = rec;\n+\t\tstruct pq_entry e = {\n+\t\t\t.rec = {\n+\t\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t\t.u.ref = {\n+\t\t\t\t\t.refname = names[i],\n+\t\t\t\t}\n+\t\t\t}\n+\t\t};\n \t\tmerged_iter_pqueue_add(&pq, e);\n \t\tmerged_iter_pqueue_check(pq);\n \t\ti = (i * 7) % N;\n@@ -55,19 +57,18 @@ static void test_pq(void)\n \n \twhile (!merged_iter_pqueue_is_empty(pq)) {\n \t\tstruct pq_entry e = merged_iter_pqueue_remove(&pq);\n-\t\tstruct reftable_ref_record *ref =\n-\t\t\treftable_record_as_ref(&e.rec);\n-\n+\t\tstruct reftable_record *rec = &e.rec;\n \t\tmerged_iter_pqueue_check(pq);\n \n+\t\tEXPECT(reftable_record_type(rec) == BLOCK_TYPE_REF);\n \t\tif (last) {\n-\t\t\tEXPECT(strcmp(last, ref->refname) < 0);\n+\t\t\tEXPECT(strcmp(last, rec->u.ref.refname) < 0);\n \t\t}\n-\t\tlast = ref->refname;\n-\t\tref->refname = NULL;\n-\t\treftable_free(ref);\n+\t\t// this is names[i], so don't dealloc.\n+\t\tlast = rec->u.ref.refname;\n+\t\trec->u.ref.refname = NULL;\n+\t\treftable_record_release(rec);\n \t}\n-\n \tfor (i = 0; i < N; i++) {\n \t\treftable_free(names[i]);\n \t}\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 8d308d858f8..4a25c1d44f4 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -239,8 +239,7 @@ static int table_iter_next_in_block(struct table_iter *ti,\n {\n \tint res = block_iter_next(&ti->bi, rec);\n \tif (res == 0 && reftable_record_type(rec) == BLOCK_TYPE_REF) {\n-\t\t((struct reftable_ref_record *)rec->data)->update_index +=\n-\t\t\tti->r->min_update_index;\n+\t\trec->u.ref.update_index += ti->r->min_update_index;\n \t}\n \n \treturn res;\n@@ -437,8 +436,7 @@ static int reader_start(struct reftable_reader *r, struct table_iter *ti,\n static int reader_seek_linear(struct reftable_reader *r, struct table_iter *ti,\n \t\t\t struct reftable_record *want)\n {\n-\tstruct reftable_record rec =\n-\t\treftable_new_record(reftable_record_type(want));\n+\tstruct reftable_record rec = reftable_new_record(reftable_record_type(want));\n \tstruct strbuf want_key = STRBUF_INIT;\n \tstruct strbuf got_key = STRBUF_INIT;\n \tstruct table_iter next = TABLE_ITER_INIT;\n@@ -475,7 +473,7 @@ static int reader_seek_linear(struct reftable_reader *r, struct table_iter *ti,\n \n done:\n \tblock_iter_close(&next.bi);\n-\treftable_record_destroy(&rec);\n+\treftable_record_release(&rec);\n \tstrbuf_release(&want_key);\n \tstrbuf_release(&got_key);\n \treturn err;\n@@ -485,34 +483,36 @@ static int reader_seek_indexed(struct reftable_reader *r,\n \t\t\t struct reftable_iterator *it,\n \t\t\t struct reftable_record *rec)\n {\n-\tstruct reftable_index_record want_index = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record want_index_rec = { NULL };\n-\tstruct reftable_index_record index_result = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record index_result_rec = { NULL };\n+\tstruct reftable_record want_index = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = { .last_key = STRBUF_INIT }\n+\t};\n+\tstruct reftable_record index_result = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = { .last_key = STRBUF_INIT },\n+\t};\n \tstruct table_iter index_iter = TABLE_ITER_INIT;\n \tstruct table_iter next = TABLE_ITER_INIT;\n \tint err = 0;\n \n-\treftable_record_key(rec, &want_index.last_key);\n-\treftable_record_from_index(&want_index_rec, &want_index);\n-\treftable_record_from_index(&index_result_rec, &index_result);\n-\n+\treftable_record_key(rec, &want_index.u.idx.last_key);\n \terr = reader_start(r, &index_iter, reftable_record_type(rec), 1);\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = reader_seek_linear(r, &index_iter, &want_index_rec);\n+\terr = reader_seek_linear(r, &index_iter, &want_index);\n \twhile (1) {\n-\t\terr = table_iter_next(&index_iter, &index_result_rec);\n+\t\terr = table_iter_next(&index_iter, &index_result);\n \t\ttable_iter_block_done(&index_iter);\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = reader_table_iter_at(r, &next, index_result.offset, 0);\n+\t\terr = reader_table_iter_at(r, &next, index_result.u.idx.offset,\n+\t\t\t\t\t 0);\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = block_iter_seek(&next.bi, &want_index.last_key);\n+\t\terr = block_iter_seek(&next.bi, &want_index.u.idx.last_key);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n@@ -540,8 +540,8 @@ static int reader_seek_indexed(struct reftable_reader *r,\n done:\n \tblock_iter_close(&next.bi);\n \ttable_iter_close(&index_iter);\n-\treftable_record_release(&want_index_rec);\n-\treftable_record_release(&index_result_rec);\n+\treftable_record_release(&want_index);\n+\treftable_record_release(&index_result);\n \treturn err;\n }\n \n@@ -549,8 +549,7 @@ static int reader_seek_internal(struct reftable_reader *r,\n \t\t\t\tstruct reftable_iterator *it,\n \t\t\t\tstruct reftable_record *rec)\n {\n-\tstruct reftable_reader_offsets *offs =\n-\t\treader_offsets_for(r, reftable_record_type(rec));\n+\tstruct reftable_reader_offsets *offs = reader_offsets_for(r, reftable_record_type(rec));\n \tuint64_t idx = offs->index_offset;\n \tstruct table_iter ti = TABLE_ITER_INIT;\n \tint err = 0;\n@@ -590,11 +589,12 @@ static int reader_seek(struct reftable_reader *r, struct reftable_iterator *it,\n int reftable_reader_seek_ref(struct reftable_reader *r,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = {\n+\t\t\t.refname = (char *)name,\n+\t\t},\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n \treturn reader_seek(r, it, &rec);\n }\n \n@@ -602,12 +602,13 @@ int reftable_reader_seek_log_at(struct reftable_reader *r,\n \t\t\t\tstruct reftable_iterator *it, const char *name,\n \t\t\t\tuint64_t update_index)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = update_index,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.u.log = {\n+\t\t\t.refname = (char *)name,\n+\t\t\t.update_index = update_index,\n+\t\t}\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n \treturn reader_seek(r, it, &rec);\n }\n \n@@ -649,31 +650,33 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \t\t\t\t\t struct reftable_iterator *it,\n \t\t\t\t\t uint8_t *oid)\n {\n-\tstruct reftable_obj_record want = {\n-\t\t.hash_prefix = oid,\n-\t\t.hash_prefix_len = r->object_id_len,\n+\tstruct reftable_record want = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.u.obj = {\n+\t\t\t.hash_prefix = oid,\n+\t\t\t.hash_prefix_len = r->object_id_len,\n+\t\t},\n \t};\n-\tstruct reftable_record want_rec = { NULL };\n \tstruct reftable_iterator oit = { NULL };\n-\tstruct reftable_obj_record got = { NULL };\n-\tstruct reftable_record got_rec = { NULL };\n+\tstruct reftable_record got = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.u.obj = { 0 },\n+\t};\n \tint err = 0;\n \tstruct indexed_table_ref_iter *itr = NULL;\n \n \t/* Look through the reverse index. */\n-\treftable_record_from_obj(&want_rec, &want);\n-\terr = reader_seek(r, &oit, &want_rec);\n+\terr = reader_seek(r, &oit, &want);\n \tif (err != 0)\n \t\tgoto done;\n \n \t/* read out the reftable_obj_record */\n-\treftable_record_from_obj(&got_rec, &got);\n-\terr = iterator_next(&oit, &got_rec);\n+\terr = iterator_next(&oit, &got);\n \tif (err < 0)\n \t\tgoto done;\n \n-\tif (err > 0 ||\n-\t memcmp(want.hash_prefix, got.hash_prefix, r->object_id_len)) {\n+\tif (err > 0 || memcmp(want.u.obj.hash_prefix, got.u.obj.hash_prefix,\n+\t\t\t r->object_id_len)) {\n \t\t/* didn't find it; return empty iterator */\n \t\titerator_set_empty(it);\n \t\terr = 0;\n@@ -681,15 +684,15 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \t}\n \n \terr = new_indexed_table_ref_iter(&itr, r, oid, hash_size(r->hash_id),\n-\t\t\t\t\t got.offsets, got.offset_len);\n+\t\t\t\t\t got.u.obj.offsets, got.u.obj.offset_len);\n \tif (err < 0)\n \t\tgoto done;\n-\tgot.offsets = NULL;\n+\tgot.u.obj.offsets = NULL;\n \titerator_from_indexed_table_ref_iter(it, itr);\n \n done:\n \treftable_iterator_destroy(&oit);\n-\treftable_record_release(&got_rec);\n+\treftable_record_release(&got);\n \treturn err;\n }\n \ndiff --git a/reftable/record.c b/reftable/record.c\nindex 8bbcbff1e69..f766ee20ace 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -15,6 +15,10 @@ https://developers.google.com/open-source/licenses/bsd\n #include \"reftable-error.h\"\n #include \"basics.h\"\n \n+static struct reftable_record_vtable *\n+reftable_record_vtable(struct reftable_record *rec);\n+static void *reftable_record_data(struct reftable_record *rec);\n+\n int get_var_int(uint64_t *dest, struct string_view *in)\n {\n \tint ptr = 0;\n@@ -926,58 +930,6 @@ static struct reftable_record_vtable reftable_log_record_vtable = {\n \t.is_deletion = &reftable_log_record_is_deletion_void,\n };\n \n-struct reftable_record reftable_new_record(uint8_t typ)\n-{\n-\tstruct reftable_record rec = { NULL };\n-\tswitch (typ) {\n-\tcase BLOCK_TYPE_REF: {\n-\t\tstruct reftable_ref_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_ref_record));\n-\t\treftable_record_from_ref(&rec, r);\n-\t\treturn rec;\n-\t}\n-\n-\tcase BLOCK_TYPE_OBJ: {\n-\t\tstruct reftable_obj_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_obj_record));\n-\t\treftable_record_from_obj(&rec, r);\n-\t\treturn rec;\n-\t}\n-\tcase BLOCK_TYPE_LOG: {\n-\t\tstruct reftable_log_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_log_record));\n-\t\treftable_record_from_log(&rec, r);\n-\t\treturn rec;\n-\t}\n-\tcase BLOCK_TYPE_INDEX: {\n-\t\tstruct reftable_index_record empty = { .last_key =\n-\t\t\t\t\t\t\t STRBUF_INIT };\n-\t\tstruct reftable_index_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_index_record));\n-\t\t*r = empty;\n-\t\treftable_record_from_index(&rec, r);\n-\t\treturn rec;\n-\t}\n-\t}\n-\tabort();\n-\treturn rec;\n-}\n-\n-/* clear out the record, yielding the reftable_record data that was\n- * encapsulated. */\n-static void *reftable_record_yield(struct reftable_record *rec)\n-{\n-\tvoid *p = rec->data;\n-\trec->data = NULL;\n-\treturn p;\n-}\n-\n-void reftable_record_destroy(struct reftable_record *rec)\n-{\n-\treftable_record_release(rec);\n-\treftable_free(reftable_record_yield(rec));\n-}\n-\n static void reftable_index_record_key(const void *r, struct strbuf *dest)\n {\n \tconst struct reftable_index_record *rec = r;\n@@ -1055,91 +1007,52 @@ static struct reftable_record_vtable reftable_index_record_vtable = {\n \n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)\n {\n-\trec->ops->key(rec->data, dest);\n+\treftable_record_vtable(rec)->key(reftable_record_data(rec), dest);\n }\n \n uint8_t reftable_record_type(struct reftable_record *rec)\n {\n-\treturn rec->ops->type;\n+\treturn rec->type;\n }\n \n int reftable_record_encode(struct reftable_record *rec, struct string_view dest,\n \t\t\t int hash_size)\n {\n-\treturn rec->ops->encode(rec->data, dest, hash_size);\n+\treturn reftable_record_vtable(rec)->encode(reftable_record_data(rec),\n+\t\t\t\t\t\t dest, hash_size);\n }\n \n void reftable_record_copy_from(struct reftable_record *rec,\n \t\t\t struct reftable_record *src, int hash_size)\n {\n-\tassert(src->ops->type == rec->ops->type);\n+\tassert(src->type == rec->type);\n \n-\trec->ops->copy_from(rec->data, src->data, hash_size);\n+\treftable_record_vtable(rec)->copy_from(reftable_record_data(rec),\n+\t\t\t\t\t reftable_record_data(src),\n+\t\t\t\t\t hash_size);\n }\n \n uint8_t reftable_record_val_type(struct reftable_record *rec)\n {\n-\treturn rec->ops->val_type(rec->data);\n+\treturn reftable_record_vtable(rec)->val_type(reftable_record_data(rec));\n }\n \n int reftable_record_decode(struct reftable_record *rec, struct strbuf key,\n \t\t\t uint8_t extra, struct string_view src, int hash_size)\n {\n-\treturn rec->ops->decode(rec->data, key, extra, src, hash_size);\n+\treturn reftable_record_vtable(rec)->decode(reftable_record_data(rec),\n+\t\t\t\t\t\t key, extra, src, hash_size);\n }\n \n void reftable_record_release(struct reftable_record *rec)\n {\n-\trec->ops->release(rec->data);\n+\treftable_record_vtable(rec)->release(reftable_record_data(rec));\n }\n \n int reftable_record_is_deletion(struct reftable_record *rec)\n {\n-\treturn rec->ops->is_deletion(rec->data);\n-}\n-\n-void reftable_record_from_ref(struct reftable_record *rec,\n-\t\t\t struct reftable_ref_record *ref_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = ref_rec;\n-\trec->ops = &reftable_ref_record_vtable;\n-}\n-\n-void reftable_record_from_obj(struct reftable_record *rec,\n-\t\t\t struct reftable_obj_record *obj_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = obj_rec;\n-\trec->ops = &reftable_obj_record_vtable;\n-}\n-\n-void reftable_record_from_index(struct reftable_record *rec,\n-\t\t\t\tstruct reftable_index_record *index_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = index_rec;\n-\trec->ops = &reftable_index_record_vtable;\n-}\n-\n-void reftable_record_from_log(struct reftable_record *rec,\n-\t\t\t struct reftable_log_record *log_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = log_rec;\n-\trec->ops = &reftable_log_record_vtable;\n-}\n-\n-struct reftable_ref_record *reftable_record_as_ref(struct reftable_record *rec)\n-{\n-\tassert(reftable_record_type(rec) == BLOCK_TYPE_REF);\n-\treturn rec->data;\n-}\n-\n-struct reftable_log_record *reftable_record_as_log(struct reftable_record *rec)\n-{\n-\tassert(reftable_record_type(rec) == BLOCK_TYPE_LOG);\n-\treturn rec->data;\n+\treturn reftable_record_vtable(rec)->is_deletion(\n+\t\treftable_record_data(rec));\n }\n \n static int hash_equal(uint8_t *a, uint8_t *b, int hash_size)\n@@ -1212,3 +1125,53 @@ void string_view_consume(struct string_view *s, int n)\n \ts->buf += n;\n \ts->len -= n;\n }\n+\n+static void *reftable_record_data(struct reftable_record *rec)\n+{\n+\tswitch (rec->type) {\n+\tcase BLOCK_TYPE_REF:\n+\t\treturn &rec->u.ref;\n+\tcase BLOCK_TYPE_LOG:\n+\t\treturn &rec->u.log;\n+\tcase BLOCK_TYPE_INDEX:\n+\t\treturn &rec->u.idx;\n+\tcase BLOCK_TYPE_OBJ:\n+\t\treturn &rec->u.obj;\n+\t}\n+\tabort();\n+}\n+\n+static struct reftable_record_vtable *\n+reftable_record_vtable(struct reftable_record *rec)\n+{\n+\tswitch (rec->type) {\n+\tcase BLOCK_TYPE_REF:\n+\t\treturn &reftable_ref_record_vtable;\n+\tcase BLOCK_TYPE_LOG:\n+\t\treturn &reftable_log_record_vtable;\n+\tcase BLOCK_TYPE_INDEX:\n+\t\treturn &reftable_index_record_vtable;\n+\tcase BLOCK_TYPE_OBJ:\n+\t\treturn &reftable_obj_record_vtable;\n+\t}\n+\tabort();\n+}\n+\n+struct reftable_record reftable_new_record(uint8_t typ)\n+{\n+\tstruct reftable_record clean_idx = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = {\n+\t\t\t.last_key = STRBUF_INIT,\n+\t\t},\n+\t};\n+\tstruct reftable_record clean = {\n+\t\t.type = typ,\n+\t};\n+\n+\tif (typ == BLOCK_TYPE_INDEX) {\n+\t\treturn clean_idx;\n+\t}\n+\n+\treturn clean;\n+}\ndiff --git a/reftable/record.h b/reftable/record.h\nindex 498e8c50bf4..407cdaba018 100644\n--- a/reftable/record.h\n+++ b/reftable/record.h\n@@ -60,16 +60,10 @@ struct reftable_record_vtable {\n \tint (*is_deletion)(const void *rec);\n };\n \n-/* record is a generic wrapper for different types of records. */\n-struct reftable_record {\n-\tvoid *data;\n-\tstruct reftable_record_vtable *ops;\n-};\n-\n /* returns true for recognized block types. Block start with the block type. */\n int reftable_is_block_type(uint8_t typ);\n \n-/* creates a malloced record of the given type. Dispose with record_destroy */\n+/* return an initialized record for the given type */\n struct reftable_record reftable_new_record(uint8_t typ);\n \n /* Encode `key` into `dest`. Sets `is_restart` to indicate a restart. Returns\n@@ -97,6 +91,22 @@ struct reftable_obj_record {\n \tint offset_len;\n };\n \n+/* record is a generic wrapper for different types of records. It is normally\n+ * created on the stack, or embedded within another struct. If the type is\n+ * known, a fresh instance can be initialized explicitly. Otherwise, use\n+ * reftable_new_record() to initialize generically (as the index_record is not\n+ * valid as 0-initialized structure)\n+ */\n+struct reftable_record {\n+\tuint8_t type;\n+\tunion {\n+\t\tstruct reftable_ref_record ref;\n+\t\tstruct reftable_log_record log;\n+\t\tstruct reftable_obj_record obj;\n+\t\tstruct reftable_index_record idx;\n+\t} u;\n+};\n+\n /* see struct record_vtable */\n \n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest);\n@@ -111,25 +121,9 @@ int reftable_record_decode(struct reftable_record *rec, struct strbuf key,\n \t\t\t int hash_size);\n int reftable_record_is_deletion(struct reftable_record *rec);\n \n-/* zeroes out the embedded record */\n+/* frees and zeroes out the embedded record */\n void reftable_record_release(struct reftable_record *rec);\n \n-/* clear and deallocate embedded record, and zero `rec`. */\n-void reftable_record_destroy(struct reftable_record *rec);\n-\n-/* initialize generic records from concrete records. The generic record should\n- * be zeroed out. */\n-void reftable_record_from_obj(struct reftable_record *rec,\n-\t\t\t struct reftable_obj_record *objrec);\n-void reftable_record_from_index(struct reftable_record *rec,\n-\t\t\t\tstruct reftable_index_record *idxrec);\n-void reftable_record_from_ref(struct reftable_record *rec,\n-\t\t\t struct reftable_ref_record *refrec);\n-void reftable_record_from_log(struct reftable_record *rec,\n-\t\t\t struct reftable_log_record *logrec);\n-struct reftable_ref_record *reftable_record_as_ref(struct reftable_record *ref);\n-struct reftable_log_record *reftable_record_as_log(struct reftable_record *ref);\n-\n /* for qsort. */\n int reftable_ref_record_compare_name(const void *a, const void *b);\n \ndiff --git a/reftable/record_test.c b/reftable/record_test.c\nindex f4ad7cace41..96f65035c1b 100644\n--- a/reftable/record_test.c\n+++ b/reftable/record_test.c\n@@ -16,24 +16,22 @@\n \n static void test_copy(struct reftable_record *rec)\n {\n-\tstruct reftable_record copy =\n-\t\treftable_new_record(reftable_record_type(rec));\n+\tstruct reftable_record copy = reftable_new_record(reftable_record_type(rec));\n+\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \t/* do it twice to catch memory leaks */\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n-\tswitch (reftable_record_type(©)) {\n+\tswitch (copy.type) {\n \tcase BLOCK_TYPE_REF:\n-\t\tEXPECT(reftable_ref_record_equal(reftable_record_as_ref(©),\n-\t\t\t\t\t\t reftable_record_as_ref(rec),\n+\t\tEXPECT(reftable_ref_record_equal(©.u.ref, &rec->u.ref,\n \t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\tbreak;\n \tcase BLOCK_TYPE_LOG:\n-\t\tEXPECT(reftable_log_record_equal(reftable_record_as_log(©),\n-\t\t\t\t\t\t reftable_record_as_log(rec),\n+\t\tEXPECT(reftable_log_record_equal(©.u.log, &rec->u.log,\n \t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\tbreak;\n \t}\n-\treftable_record_destroy(©);\n+\treftable_record_release(©);\n }\n \n static void test_varint_roundtrip(void)\n@@ -106,61 +104,60 @@ static void test_reftable_ref_record_roundtrip(void)\n \tint i = 0;\n \n \tfor (i = REFTABLE_REF_DELETION; i < REFTABLE_NR_REF_VALUETYPES; i++) {\n-\t\tstruct reftable_ref_record in = { NULL };\n-\t\tstruct reftable_ref_record out = { NULL };\n-\t\tstruct reftable_record rec_out = { NULL };\n+\t\tstruct reftable_record in = {\n+\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t};\n+\t\tstruct reftable_record out = {\n+\t\t\t.type = BLOCK_TYPE_REF\n+\t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n-\t\tstruct reftable_record rec = { NULL };\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n \t\t\t.buf = buffer,\n \t\t\t.len = sizeof(buffer),\n \t\t};\n-\n \t\tint n, m;\n \n-\t\tin.value_type = i;\n+\t\tin.u.ref.value_type = i;\n \t\tswitch (i) {\n \t\tcase REFTABLE_REF_DELETION:\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_VAL1:\n-\t\t\tin.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val1, 1);\n+\t\t\tin.u.ref.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n+\t\t\tset_hash(in.u.ref.value.val1, 1);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_VAL2:\n-\t\t\tin.value.val2.value = reftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val2.value, 1);\n-\t\t\tin.value.val2.target_value =\n+\t\t\tin.u.ref.value.val2.value =\n+\t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n+\t\t\tset_hash(in.u.ref.value.val2.value, 1);\n+\t\t\tin.u.ref.value.val2.target_value =\n \t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val2.target_value, 2);\n+\t\t\tset_hash(in.u.ref.value.val2.target_value, 2);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_SYMREF:\n-\t\t\tin.value.symref = xstrdup(\"target\");\n+\t\t\tin.u.ref.value.symref = xstrdup(\"target\");\n \t\t\tbreak;\n \t\t}\n-\t\tin.refname = xstrdup(\"refs/heads/master\");\n+\t\tin.u.ref.refname = xstrdup(\"refs/heads/master\");\n \n-\t\treftable_record_from_ref(&rec, &in);\n-\t\ttest_copy(&rec);\n+\t\ttest_copy(&in);\n \n-\t\tEXPECT(reftable_record_val_type(&rec) == i);\n+\t\tEXPECT(reftable_record_val_type(&in) == i);\n \n-\t\treftable_record_key(&rec, &key);\n-\t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\t\treftable_record_key(&in, &key);\n+\t\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n > 0);\n \n \t\t/* decode into a non-zero reftable_record to test for leaks. */\n-\n-\t\treftable_record_from_ref(&rec_out, &out);\n-\t\tm = reftable_record_decode(&rec_out, key, i, dest,\n-\t\t\t\t\t GIT_SHA1_RAWSZ);\n+\t\tm = reftable_record_decode(&out, key, i, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_ref_record_equal(&in, &out, GIT_SHA1_RAWSZ));\n-\t\treftable_record_release(&rec_out);\n+\t\tEXPECT(reftable_ref_record_equal(&in.u.ref, &out.u.ref,\n+\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n+\t\treftable_record_release(&in);\n \n \t\tstrbuf_release(&key);\n-\t\treftable_ref_record_release(&in);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n@@ -213,7 +210,9 @@ static void test_reftable_log_record_roundtrip(void)\n \tset_test_hash(in[0].value.update.new_hash, 1);\n \tset_test_hash(in[0].value.update.old_hash, 2);\n \tfor (i = 0; i < ARRAY_SIZE(in); i++) {\n-\t\tstruct reftable_record rec = { NULL };\n+\t\tstruct reftable_record rec = {\n+\t\t\t.type = BLOCK_TYPE_LOG\n+\t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n@@ -221,23 +220,25 @@ static void test_reftable_log_record_roundtrip(void)\n \t\t\t.len = sizeof(buffer),\n \t\t};\n \t\t/* populate out, to check for leaks. */\n-\t\tstruct reftable_log_record out = {\n-\t\t\t.refname = xstrdup(\"old name\"),\n-\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n-\t\t\t.value = {\n-\t\t\t\t.update = {\n-\t\t\t\t\t.new_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n-\t\t\t\t\t.old_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n-\t\t\t\t\t.name = xstrdup(\"old name\"),\n-\t\t\t\t\t.email = xstrdup(\"old@email\"),\n-\t\t\t\t\t.message = xstrdup(\"old message\"),\n+\t\tstruct reftable_record out = {\n+\t\t\t.type = BLOCK_TYPE_LOG,\n+\t\t\t.u.log = {\n+\t\t\t\t.refname = xstrdup(\"old name\"),\n+\t\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n+\t\t\t\t.value = {\n+\t\t\t\t\t.update = {\n+\t\t\t\t\t\t.new_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t\t.old_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t\t.name = xstrdup(\"old name\"),\n+\t\t\t\t\t\t.email = xstrdup(\"old@email\"),\n+\t\t\t\t\t\t.message = xstrdup(\"old message\"),\n+\t\t\t\t\t},\n \t\t\t\t},\n \t\t\t},\n \t\t};\n-\t\tstruct reftable_record rec_out = { NULL };\n \t\tint n, m, valtype;\n \n-\t\treftable_record_from_log(&rec, &in[i]);\n+\t\trec.u.log = in[i];\n \n \t\ttest_copy(&rec);\n \n@@ -245,16 +246,16 @@ static void test_reftable_log_record_roundtrip(void)\n \n \t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n >= 0);\n-\t\treftable_record_from_log(&rec_out, &out);\n \t\tvaltype = reftable_record_val_type(&rec);\n-\t\tm = reftable_record_decode(&rec_out, key, valtype, dest,\n+\t\tm = reftable_record_decode(&out, key, valtype, dest,\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_log_record_equal(&in[i], &out, GIT_SHA1_RAWSZ));\n+\t\tEXPECT(reftable_log_record_equal(&in[i], &out.u.log,\n+\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\treftable_log_record_release(&in[i]);\n \t\tstrbuf_release(&key);\n-\t\treftable_record_release(&rec_out);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n@@ -322,47 +323,52 @@ static void test_reftable_obj_record_roundtrip(void)\n \t\t\t\t\t } };\n \tint i = 0;\n \tfor (i = 0; i < ARRAY_SIZE(recs); i++) {\n-\t\tstruct reftable_obj_record in = recs[i];\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n \t\t\t.buf = buffer,\n \t\t\t.len = sizeof(buffer),\n \t\t};\n-\t\tstruct reftable_record rec = { NULL };\n+\t\tstruct reftable_record in = {\n+\t\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t\t.u.obj = recs[i],\n+\t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n-\t\tstruct reftable_obj_record out = { NULL };\n-\t\tstruct reftable_record rec_out = { NULL };\n+\t\tstruct reftable_record out = {\n+\t\t\t.type = BLOCK_TYPE_OBJ\n+\t\t};\n \t\tint n, m;\n \t\tuint8_t extra;\n \n-\t\treftable_record_from_obj(&rec, &in);\n-\t\ttest_copy(&rec);\n-\t\treftable_record_key(&rec, &key);\n-\t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\t\ttest_copy(&in);\n+\t\treftable_record_key(&in, &key);\n+\t\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n > 0);\n-\t\textra = reftable_record_val_type(&rec);\n-\t\treftable_record_from_obj(&rec_out, &out);\n-\t\tm = reftable_record_decode(&rec_out, key, extra, dest,\n+\t\textra = reftable_record_val_type(&in);\n+\t\tm = reftable_record_decode(&out, key, extra, dest,\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(in.hash_prefix_len == out.hash_prefix_len);\n-\t\tEXPECT(in.offset_len == out.offset_len);\n-\n-\t\tEXPECT(!memcmp(in.hash_prefix, out.hash_prefix,\n-\t\t\t in.hash_prefix_len));\n-\t\tEXPECT(0 == memcmp(in.offsets, out.offsets,\n-\t\t\t\t sizeof(uint64_t) * in.offset_len));\n+\t\tEXPECT(in.u.obj.hash_prefix_len == out.u.obj.hash_prefix_len);\n+\t\tEXPECT(in.u.obj.offset_len == out.u.obj.offset_len);\n+\t\tif (in.u.obj.hash_prefix_len)\n+\t\t\tEXPECT(!memcmp(in.u.obj.hash_prefix, out.u.obj.hash_prefix,\n+\t\t\t\t in.u.obj.hash_prefix_len));\n+\t\tif (in.u.obj.offset_len)\n+\t\t\tEXPECT(!memcmp(in.u.obj.offsets, out.u.obj.offsets,\n+\t\t\t\t\t sizeof(uint64_t) * in.u.obj.offset_len));\n \t\tstrbuf_release(&key);\n-\t\treftable_record_release(&rec_out);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n static void test_reftable_index_record_roundtrip(void)\n {\n-\tstruct reftable_index_record in = {\n-\t\t.offset = 42,\n-\t\t.last_key = STRBUF_INIT,\n+\tstruct reftable_record in = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = {\n+\t\t\t.offset = 42,\n+\t\t\t.last_key = STRBUF_INIT,\n+\t\t},\n \t};\n \tuint8_t buffer[1024] = { 0 };\n \tstruct string_view dest = {\n@@ -370,31 +376,30 @@ static void test_reftable_index_record_roundtrip(void)\n \t\t.len = sizeof(buffer),\n \t};\n \tstruct strbuf key = STRBUF_INIT;\n-\tstruct reftable_record rec = { NULL };\n-\tstruct reftable_index_record out = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record out_rec = { NULL };\n+\tstruct reftable_record out = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = { .last_key = STRBUF_INIT },\n+\t};\n \tint n, m;\n \tuint8_t extra;\n \n-\tstrbuf_addstr(&in.last_key, \"refs/heads/master\");\n-\treftable_record_from_index(&rec, &in);\n-\treftable_record_key(&rec, &key);\n-\ttest_copy(&rec);\n+\tstrbuf_addstr(&in.u.idx.last_key, \"refs/heads/master\");\n+\treftable_record_key(&in, &key);\n+\ttest_copy(&in);\n \n-\tEXPECT(0 == strbuf_cmp(&key, &in.last_key));\n-\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\tEXPECT(0 == strbuf_cmp(&key, &in.u.idx.last_key));\n+\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(n > 0);\n \n-\textra = reftable_record_val_type(&rec);\n-\treftable_record_from_index(&out_rec, &out);\n-\tm = reftable_record_decode(&out_rec, key, extra, dest, GIT_SHA1_RAWSZ);\n+\textra = reftable_record_val_type(&in);\n+\tm = reftable_record_decode(&out, key, extra, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(m == n);\n \n-\tEXPECT(in.offset == out.offset);\n+\tEXPECT(in.u.idx.offset == out.u.idx.offset);\n \n-\treftable_record_release(&out_rec);\n+\treftable_record_release(&out);\n \tstrbuf_release(&key);\n-\tstrbuf_release(&in.last_key);\n+\tstrbuf_release(&in.u.idx.last_key);\n }\n \n int record_test_main(int argc, const char *argv[])\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex 3ca721e9f64..990f78ff7ed 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -251,8 +251,10 @@ done:\n int reftable_writer_add_ref(struct reftable_writer *w,\n \t\t\t struct reftable_ref_record *ref)\n {\n-\tstruct reftable_record rec = { NULL };\n-\tstruct reftable_ref_record copy = *ref;\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = *ref,\n+\t};\n \tint err = 0;\n \n \tif (ref->refname == NULL)\n@@ -261,8 +263,7 @@ int reftable_writer_add_ref(struct reftable_writer *w,\n \t ref->update_index > w->max_update_index)\n \t\treturn REFTABLE_API_ERROR;\n \n-\treftable_record_from_ref(&rec, ©);\n-\tcopy.update_index -= w->min_update_index;\n+\trec.u.ref.update_index -= w->min_update_index;\n \n \terr = writer_add_record(w, &rec);\n \tif (err < 0)\n@@ -301,7 +302,10 @@ int reftable_writer_add_refs(struct reftable_writer *w,\n static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n \t\t\t\t\t struct reftable_log_record *log)\n {\n-\tstruct reftable_record rec = { NULL };\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.u.log = *log,\n+\t};\n \tif (w->block_writer &&\n \t block_writer_type(w->block_writer) == BLOCK_TYPE_REF) {\n \t\tint err = writer_finish_public_section(w);\n@@ -311,8 +315,6 @@ static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n \n \tw->next -= w->pending_padding;\n \tw->pending_padding = 0;\n-\n-\treftable_record_from_log(&rec, log);\n \treturn writer_add_record(w, &rec);\n }\n \n@@ -393,8 +395,10 @@ static int writer_finish_section(struct reftable_writer *w)\n \t\tw->index_len = 0;\n \t\tw->index_cap = 0;\n \t\tfor (i = 0; i < idx_len; i++) {\n-\t\t\tstruct reftable_record rec = { NULL };\n-\t\t\treftable_record_from_index(&rec, idx + i);\n+\t\t\tstruct reftable_record rec = {\n+\t\t\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t\t\t.u.idx = idx[i],\n+\t\t\t};\n \t\t\tif (block_writer_add(w->block_writer, &rec) == 0) {\n \t\t\t\tcontinue;\n \t\t\t}\n@@ -462,17 +466,18 @@ static void write_object_record(void *void_arg, void *key)\n {\n \tstruct write_record_arg *arg = void_arg;\n \tstruct obj_index_tree_node *entry = key;\n-\tstruct reftable_obj_record obj_rec = {\n-\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n-\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n-\t\t.offsets = entry->offsets,\n-\t\t.offset_len = entry->offset_len,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.u.obj = {\n+\t\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n+\t\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n+\t\t\t.offsets = entry->offsets,\n+\t\t\t.offset_len = entry->offset_len,\n+\t\t}\n \t};\n-\tstruct reftable_record rec = { NULL };\n \tif (arg->err < 0)\n \t\tgoto done;\n \n-\treftable_record_from_obj(&rec, &obj_rec);\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \tif (arg->err == 0)\n \t\tgoto done;\n@@ -485,7 +490,8 @@ static void write_object_record(void *void_arg, void *key)\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \tif (arg->err == 0)\n \t\tgoto done;\n-\tobj_rec.offset_len = 0;\n+\n+\trec.u.obj.offset_len = 0;\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \n \t/* Should be able to write into a fresh block. */\n-- \ngitgitgadget\n"},{"id":"444002","messageId":"211213.86pmq08ogt.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"975a570d388fca79546987f4683fcd33419aad98.1639411309.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 02/11] reftable: fix resource leak in error path","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-12-13T16:19:29Z","receivedAt":"2021-12-13T16:22:02Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Dec 13 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n> [...]\n> -\treturn 0;\n> +done:\n> +\tif (uncompressed) {\n> +\t\treftable_free(uncompressed);\n> +\t}\n> +\treturn err;\n> }\n\nOther things in the codebase don't check for NULL before feeding things\nto reftable_free(), and our own free() has a coccicheck rule to catch\nthis sort of code, we should probably add reftable_free to that list...\n\n> \n> static uint32_t block_reader_restart_offset(struct block_reader *br, int i)\n> diff --git a/reftable/readwrite_test.c b/reftable/readwrite_test.c\n> index 5f6bcc2f775..42caf0bde4c 100644\n> --- a/reftable/readwrite_test.c\n> +++ b/reftable/readwrite_test.c\n> @@ -254,6 +254,72 @@ static void test_log_write_read(void)\n> \treader_close(&rd);\n> }\n> \n> +static void test_log_zlib_corruption(void)\n> +{\n> +\tstruct reftable_write_options opts = {\n> +\t\t.block_size = 256,\n> +\t};\n> +\tstruct reftable_iterator it = { NULL };\n> +\tstruct reftable_reader rd = { NULL };\n> +\tstruct reftable_block_source source = { NULL };\n\nNit: It doesn't matter for semantics, but usually we use \"{ 0 }\", and\nyour 1/11 does too. Would be better to do that here for consistency.\n\n> +\tfor (i = 0; i < sizeof(message)-1; i++) {\n> +\t\tmessage[i] = (uint8_t)(rand() % 64 + ' ');\n> +\t}\n\nstyle: braces not needede.\n"},{"id":"444003","messageId":"211213.86lf0o8ob7.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"2939286924caa67e2cc57bf5ed4ce4287b9d571c.1639411309.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 07/11] reftable: fix NULL derefs in error paths","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-12-13T16:24:54Z","receivedAt":"2021-12-13T16:25:21Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Dec 13 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> Spotted by Coverity.\n>\n> Signed-off-by: Han-Wen Nienhuys \n> ---\n> reftable/reader.c | 3 ++-\n> reftable/stack.c | 3 ++-\n> 2 files changed, 4 insertions(+), 2 deletions(-)\n>\n> diff --git a/reftable/reader.c b/reftable/reader.c\n> index 006709a645a..8d308d858f8 100644\n> --- a/reftable/reader.c\n> +++ b/reftable/reader.c\n> @@ -796,6 +796,7 @@ int reftable_reader_print_file(const char *tablename)\n> \treftable_table_from_reader(&tab, r);\n> \terr = reftable_table_print(&tab);\n> done:\n> -\treftable_reader_free(r);\n> +\tif (r)\n> +\t\treftable_reader_free(r);\n> \treturn err;\n> }\n> diff --git a/reftable/stack.c b/reftable/stack.c\n> index 10dfd370e8e..eb03b6c74f6 100644\n> --- a/reftable/stack.c\n> +++ b/reftable/stack.c\n> @@ -707,7 +707,8 @@ done:\n> \tstrbuf_release(&temp_tab_file_name);\n> \tstrbuf_release(&tab_file_name);\n> \tstrbuf_release(&next_name);\n> -\treftable_writer_free(wr);\n> +\tif (wr)\n> +\t\treftable_writer_free(wr);\n> \treturn err;\n> }\n\nI'd think this would be better:\n\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 006709a645a..3eec915830d 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -641,7 +641,8 @@ int reftable_new_reader(struct reftable_reader **p,\n \n void reftable_reader_free(struct reftable_reader *r)\n {\n- reader_close(r);\n+ if (r)\n+ reader_close(r);\n reftable_free(r);\n }\n\nNo?\n"},{"id":"444004","messageId":"211213.86h7bc8o9i.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"9dce18d7349fe2a27859c224068fb5a386f263f1.1639411309.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 08/11] reftable: order unittests by complexity","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-12-13T16:25:23Z","receivedAt":"2021-12-13T16:26:22Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Dec 13 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> This is a more pratical ordering when working on refactorings of the reftable\n> code.\n>\n> Signed-off-by: Han-Wen Nienhuys \n> ---\n> t/helper/test-reftable.c | 9 +++++----\n> 1 file changed, 5 insertions(+), 4 deletions(-)\n>\n> diff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c\n> index 26b03d7b789..f08d66df949 100644\n> --- a/t/helper/test-reftable.c\n> +++ b/t/helper/test-reftable.c\n> @@ -3,15 +3,16 @@\n> \n> int cmd__reftable(int argc, const char **argv)\n> {\n> +\t// test from simple to complex.\n\nA C99 comment, which I don't mind, but is explicitly forbidden by the\nstyle guide (and when I mentioned recently that we didn't have any\nportability reasons not to use these, there wasn't much/any interest in\nchanging that...).\n"},{"id":"444005","messageId":"211213.86czm08o8k.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"6b0af68f0b94fb96c81e25f9a1911fa05dcf07fd.1639411309.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 09/11] reftable: drop stray printf in readwrite_test","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-12-13T16:26:31Z","receivedAt":"2021-12-13T16:27:02Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Dec 13 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> Signed-off-by: Han-Wen Nienhuys \n> ---\n> reftable/readwrite_test.c | 1 -\n> 1 file changed, 1 deletion(-)\n>\n> diff --git a/reftable/readwrite_test.c b/reftable/readwrite_test.c\n> index 42caf0bde4c..f6546d7b8d1 100644\n> --- a/reftable/readwrite_test.c\n> +++ b/reftable/readwrite_test.c\n> @@ -663,7 +663,6 @@ static void test_write_key_order(void)\n> \terr = reftable_writer_add_ref(w, &refs[0]);\n> \tEXPECT_ERR(err);\n> \terr = reftable_writer_add_ref(w, &refs[1]);\n> -\tprintf(\"%d\\n\", err);\n> \tEXPECT(err == REFTABLE_API_ERROR);\n> \treftable_writer_close(w);\n> \treftable_writer_free(w);\n\nIs this something coverity raised and I'm missing why it's bad (per the\nCL), or just a stray unrelated cleanup?\n"},{"id":"444007","messageId":"CAFQ2z_MrSEExScfBPku1uTZQgBYyk-B08U98zs5P7LNdLphutg@mail.gmail.com","threadId":"57047","inReplyTo":"211213.86pmq08ogt.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v3 02/11] reftable: fix resource leak in error path","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-12-13T16:44:46Z","receivedAt":"2021-12-13T16:44:59Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Mon, Dec 13, 2021 at 5:21 PM Ævar Arnfjörð Bjarmason\n wrote:\n> Other things in the codebase don't check for NULL before feeding things\n> to reftable_free(), and our own free() has a coccicheck rule to catch\n> this sort of code, we should probably add reftable_free to that list...\n\nThanks, changed.\n\n> > + struct reftable_block_source source = { NULL };\n>\n> Nit: It doesn't matter for semantics, but usually we use \"{ 0 }\", and\n> your 1/11 does too. Would be better to do that here for consistency.\n\nI got a past review where someone complained about this. I don't mind\neither way, but would rather not flipflop.\n\n>\n> > + for (i = 0; i < sizeof(message)-1; i++) {\n> > + message[i] = (uint8_t)(rand() % 64 + ' ');\n> > + }\n>\n> style: braces not needede.\n\nDone.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"444008","messageId":"CAFQ2z_NFXEWe5rMgDuao+fdOio3tV_33ViwZaZnc5u2LgLxEBg@mail.gmail.com","threadId":"57047","inReplyTo":"211213.86h7bc8o9i.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v3 08/11] reftable: order unittests by complexity","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-12-13T16:45:30Z","receivedAt":"2021-12-13T16:45:43Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Mon, Dec 13, 2021 at 5:26 PM Ævar Arnfjörð Bjarmason\n wrote:\n> A C99 comment, which I don't mind, but is explicitly forbidden by the\n> style guide (and when I mentioned recently that we didn't have any\n> portability reasons not to use these, there wasn't much/any interest in\n> changing that...).\n\nsorry, I missed this comment in a previous round. Fixed now.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"444010","messageId":"CAFQ2z_Nr-Rf2kjujPcMtRV6VfweP68A44ZD2XPL8+CqM4F4hZg@mail.gmail.com","threadId":"57047","inReplyTo":"211213.86czm08o8k.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v3 09/11] reftable: drop stray printf in readwrite_test","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-12-13T16:46:27Z","receivedAt":"2021-12-13T16:46:41Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Mon, Dec 13, 2021 at 5:26 PM Ævar Arnfjörð Bjarmason\n wrote:\n> Is this something coverity raised and I'm missing why it's bad (per the\n> CL), or just a stray unrelated cleanup?\n\nI spotted it while working on this series. It prints something\nunexpected into the output of the unittest, so better remove it.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"444040","messageId":"xmqqzgp49mwn.fsf@gitster.g","threadId":"57047","inReplyTo":"CAFQ2z_MrSEExScfBPku1uTZQgBYyk-B08U98zs5P7LNdLphutg@mail.gmail.com","subject":"Re: [PATCH v3 02/11] reftable: fix resource leak in error path","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-13T22:10:16Z","receivedAt":"2021-12-13T22:10:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Han-Wen Nienhuys writes:\n\n> On Mon, Dec 13, 2021 at 5:21 PM Ævar Arnfjörð Bjarmason\n> wrote:\n>> Other things in the codebase don't check for NULL before feeding things\n>> to reftable_free(), and our own free() has a coccicheck rule to catch\n>> this sort of code, we should probably add reftable_free to that list...\n>\n> Thanks, changed.\n>\n>> > + struct reftable_block_source source = { NULL };\n>>\n>> Nit: It doesn't matter for semantics, but usually we use \"{ 0 }\", and\n>> your 1/11 does too. Would be better to do that here for consistency.\n>\n> I got a past review where someone complained about this. I don't mind\n> either way, but would rather not flipflop.\n\nThe last part is important. \n\nFor initializers that show the value for the first member in {} to\nmean \"everything is zero-initialized\", only because the language\ndoes not allow us to write\n\n\tstruct foo var = {};\n\nwe historically used { NULL } for pointers and { 0 } for integral\ntypes (primarily because auto checkers like sparse did not like us\nto write a NULL pointer as 0), but we started preferring { 0 } as\nmore recent versions of checkers understand it as an idiom, and we\ncan freely reorder the struct members if we consistently spell the\n\"everything is zero-initialized\" that way.\n\nSo, let's use { 0 } here, too.\n\nThanks.\n"},{"id":"444041","messageId":"xmqqv8zs9mrr.fsf@gitster.g","threadId":"57047","inReplyTo":"211213.86h7bc8o9i.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v3 08/11] reftable: order unittests by complexity","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-13T22:13:12Z","receivedAt":"2021-12-13T22:13:19Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n>> int cmd__reftable(int argc, const char **argv)\n>> {\n>> +\t// test from simple to complex.\n>\n> A C99 comment, which I don't mind, but is explicitly forbidden by the\n> style guide (and when I mentioned recently that we didn't have any\n> portability reasons not to use these, there wasn't much/any interest in\n> changing that...).\n\nUnlike the decl-after-statement thing, I personally do not mind\nsomething that does not upset compilers and does not hurt human\nreadability greatly, like the use of \"//\". But I do not care about\nit deeply enough to find it worth my time to raise it as a separate\ntopic and update the guideline.\n\n"},{"id":"444086","messageId":"pull.1152.v4.git.git.1639482476.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v3.git.git.1639411309.gitgitgadget@gmail.com","subject":"[PATCH v4 00/11] Reftable coverity fixes","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-14T11:47:45Z","receivedAt":"2021-12-14T11:48:01Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"This series was targeted to 'next'.\n\nThis addresses some complaints from Coverity that Peff reported.\n\nv3:\n\n * revert some changes suggested by stolee\n * add unittest for zlib corruption.\n\nHan-Wen Nienhuys (11):\n reftable: fix OOB stack write in print functions\n reftable: fix resource leak in block.c error path\n reftable: fix resource leak blocksource.c\n reftable: check reftable_stack_auto_compact() return value\n reftable: ignore remove() return value in stack_test.c\n reftable: fix resource warning\n reftable: all xxx_free() functions accept NULL arguments\n reftable: order unittests by complexity\n reftable: drop stray printf in readwrite_test\n reftable: handle null refnames in reftable_ref_record_equal\n reftable: make reftable_record a tagged union\n\n reftable/block.c | 28 +++---\n reftable/block_test.c | 22 ++---\n reftable/blocksource.c | 6 +-\n reftable/generic.c | 35 ++++----\n reftable/iter.c | 4 +-\n reftable/merged.c | 33 +++----\n reftable/pq.c | 3 +-\n reftable/pq_test.c | 31 +++----\n reftable/reader.c | 95 ++++++++++----------\n reftable/readwrite_test.c | 67 +++++++++++++-\n reftable/record.c | 185 ++++++++++++++++----------------------\n reftable/record.h | 42 ++++-----\n reftable/record_test.c | 181 +++++++++++++++++++------------------\n reftable/stack.c | 12 +--\n reftable/stack_test.c | 3 +-\n reftable/writer.c | 42 +++++----\n t/helper/test-reftable.c | 9 +-\n 17 files changed, 428 insertions(+), 370 deletions(-)\n\n\nbase-commit: fae76fe5da3df25d752f2251b7ccda3f62813aa9\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1152%2Fhanwen%2Freftable-coverity-v4\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1152/hanwen/reftable-coverity-v4\nPull-Request: https://github.com/git/git/pull/1152\n\nRange-diff vs v3:\n\n 1: 1c1a3ff92bd = 1: 1c1a3ff92bd reftable: fix OOB stack write in print functions\n 2: 975a570d388 ! 2: 923293b79b4 reftable: fix resource leak in error path\n @@ Metadata\n Author: Han-Wen Nienhuys \n \n ## Commit message ##\n - reftable: fix resource leak in error path\n + reftable: fix resource leak in block.c error path\n \n Add test coverage for corrupt zlib data.\n \n @@ reftable/block.c: int block_reader_init(struct block_reader *br, struct reftable\n \n -\treturn 0;\n +done:\n -+\tif (uncompressed) {\n -+\t\treftable_free(uncompressed);\n -+\t}\n ++\treftable_free(uncompressed);\n +\treturn err;\n }\n \n @@ reftable/readwrite_test.c: static void test_log_write_read(void)\n +\tstruct reftable_write_options opts = {\n +\t\t.block_size = 256,\n +\t};\n -+\tstruct reftable_iterator it = { NULL };\n -+\tstruct reftable_reader rd = { NULL };\n -+\tstruct reftable_block_source source = { NULL };\n ++\tstruct reftable_iterator it = { 0 };\n ++\tstruct reftable_reader rd = { 0 };\n ++\tstruct reftable_block_source source = { 0 };\n +\tstruct strbuf buf = STRBUF_INIT;\n +\tstruct reftable_writer *w =\n +\t\treftable_new_writer(&strbuf_add_void, &buf, &opts);\n @@ reftable/readwrite_test.c: static void test_log_write_read(void)\n +\t\t},\n +\t};\n +\n -+\tfor (i = 0; i < sizeof(message)-1; i++) {\n ++\tfor (i = 0; i < sizeof(message)-1; i++)\n +\t\tmessage[i] = (uint8_t)(rand() % 64 + ' ');\n -+\t}\n +\n +\treftable_writer_set_limits(w, 1, 1);\n +\n 3: 0b9c7176d71 = 3: 7a914f77756 reftable: fix resource leak blocksource.c\n 4: 1dda4ee717f = 4: 51b4a17a2e1 reftable: check reftable_stack_auto_compact() return value\n 5: 36858e2070b = 5: 43989afcb5a reftable: ignore remove() return value in stack_test.c\n 6: 80b1988b885 = 6: 1c7f15d811c reftable: fix resource warning\n 7: 2939286924c < -: ----------- reftable: fix NULL derefs in error paths\n -: ----------- > 7: 47ba5ddceb8 reftable: all xxx_free() functions accept NULL arguments\n 8: 9dce18d7349 ! 8: aba8b8113ad reftable: order unittests by complexity\n @@ Metadata\n ## Commit message ##\n reftable: order unittests by complexity\n \n - This is a more pratical ordering when working on refactorings of the reftable\n + This is a more practical ordering when working on refactorings of the reftable\n code.\n \n Signed-off-by: Han-Wen Nienhuys \n @@ t/helper/test-reftable.c\n \n int cmd__reftable(int argc, const char **argv)\n {\n -+\t// test from simple to complex.\n ++\t/* test from simple to complex. */\n \tbasics_test_main(argc, argv);\n +\trecord_test_main(argc, argv);\n \tblock_test_main(argc, argv);\n 9: 6b0af68f0b9 = 9: f2af404d7f1 reftable: drop stray printf in readwrite_test\n 10: bff85cb0809 = 10: d2ce1bfef9f reftable: handle null refnames in reftable_ref_record_equal\n 11: b3e592b9c27 = 11: 0ffc1cf0e42 reftable: make reftable_record a tagged union\n\n-- \ngitgitgadget\n"},{"id":"444087","messageId":"1c1a3ff92bd24f7c588ce480983cecb3508a0add.1639482477.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v4.git.git.1639482476.gitgitgadget@gmail.com","subject":"[PATCH v4 01/11] reftable: fix OOB stack write in print functions","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-14T11:47:46Z","receivedAt":"2021-12-14T11:48:04Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 6a5dac32dc6..8536bd03aa9 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -254,7 +254,7 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)\n void reftable_ref_record_print(struct reftable_ref_record *ref,\n \t\t\t uint32_t hash_id)\n {\n-\tchar hex[2 * GIT_SHA256_RAWSZ + 1] = { 0 }; /* BUG */\n+\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */\n \tprintf(\"ref{%s(%\" PRIu64 \") \", ref->refname, ref->update_index);\n \tswitch (ref->value_type) {\n \tcase REFTABLE_REF_SYMREF:\n@@ -586,7 +586,7 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {\n void reftable_log_record_print(struct reftable_log_record *log,\n \t\t\t uint32_t hash_id)\n {\n-\tchar hex[GIT_SHA256_RAWSZ + 1] = { 0 };\n+\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 };\n \n \tswitch (log->value_type) {\n \tcase REFTABLE_LOG_DELETION:\n-- \ngitgitgadget\n\n"},{"id":"444088","messageId":"923293b79b4a450df1b6577a0e6a696496ce46db.1639482477.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v4.git.git.1639482476.gitgitgadget@gmail.com","subject":"[PATCH v4 02/11] reftable: fix resource leak in block.c error path","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-14T11:47:47Z","receivedAt":"2021-12-14T11:48:11Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nAdd test coverage for corrupt zlib data.\n\nThis problem was discovered by a Coverity scan.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/block.c | 26 +++++++++------\n reftable/readwrite_test.c | 66 +++++++++++++++++++++++++++++++++++++++\n 2 files changed, 83 insertions(+), 9 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 855e3f5c947..6c8e8705205 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -188,13 +188,16 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tuint32_t full_block_size = table_block_size;\n \tuint8_t typ = block->data[header_off];\n \tuint32_t sz = get_be24(block->data + header_off + 1);\n-\n+\tint err = 0;\n \tuint16_t restart_count = 0;\n \tuint32_t restart_start = 0;\n \tuint8_t *restart_bytes = NULL;\n+\tuint8_t *uncompressed = NULL;\n \n-\tif (!reftable_is_block_type(typ))\n-\t\treturn REFTABLE_FORMAT_ERROR;\n+\tif (!reftable_is_block_type(typ)) {\n+\t\terr = REFTABLE_FORMAT_ERROR;\n+\t\tgoto done;\n+\t}\n \n \tif (typ == BLOCK_TYPE_LOG) {\n \t\tint block_header_skip = 4 + header_off;\n@@ -203,7 +206,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\tuLongf src_len = block->len - block_header_skip;\n \t\t/* Log blocks specify the *uncompressed* size in their header.\n \t\t */\n-\t\tuint8_t *uncompressed = reftable_malloc(sz);\n+\t\tuncompressed = reftable_malloc(sz);\n \n \t\t/* Copy over the block header verbatim. It's not compressed. */\n \t\tmemcpy(uncompressed, block->data, block_header_skip);\n@@ -212,16 +215,19 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\tif (Z_OK !=\n \t\t uncompress2(uncompressed + block_header_skip, &dst_len,\n \t\t\t\tblock->data + block_header_skip, &src_len)) {\n-\t\t\treftable_free(uncompressed);\n-\t\t\treturn REFTABLE_ZLIB_ERROR;\n+\t\t\terr = REFTABLE_ZLIB_ERROR;\n+\t\t\tgoto done;\n \t\t}\n \n-\t\tif (dst_len + block_header_skip != sz)\n-\t\t\treturn REFTABLE_FORMAT_ERROR;\n+\t\tif (dst_len + block_header_skip != sz) {\n+\t\t\terr = REFTABLE_FORMAT_ERROR;\n+\t\t\tgoto done;\n+\t\t}\n \n \t\t/* We're done with the input data. */\n \t\treftable_block_done(block);\n \t\tblock->data = uncompressed;\n+\t\tuncompressed = NULL;\n \t\tblock->len = sz;\n \t\tblock->source = malloc_block_source();\n \t\tfull_block_size = src_len + block_header_skip;\n@@ -251,7 +257,9 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tbr->restart_count = restart_count;\n \tbr->restart_bytes = restart_bytes;\n \n-\treturn 0;\n+done:\n+\treftable_free(uncompressed);\n+\treturn err;\n }\n \n static uint32_t block_reader_restart_offset(struct block_reader *br, int i)\ndiff --git a/reftable/readwrite_test.c b/reftable/readwrite_test.c\nindex 5f6bcc2f775..6e88182a83a 100644\n--- a/reftable/readwrite_test.c\n+++ b/reftable/readwrite_test.c\n@@ -254,6 +254,71 @@ static void test_log_write_read(void)\n \treader_close(&rd);\n }\n \n+static void test_log_zlib_corruption(void)\n+{\n+\tstruct reftable_write_options opts = {\n+\t\t.block_size = 256,\n+\t};\n+\tstruct reftable_iterator it = { 0 };\n+\tstruct reftable_reader rd = { 0 };\n+\tstruct reftable_block_source source = { 0 };\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tstruct reftable_writer *w =\n+\t\treftable_new_writer(&strbuf_add_void, &buf, &opts);\n+\tconst struct reftable_stats *stats = NULL;\n+\tuint8_t hash1[GIT_SHA1_RAWSZ] = { 1 };\n+\tuint8_t hash2[GIT_SHA1_RAWSZ] = { 2 };\n+\tchar message[100] = { 0 };\n+\tint err, i, n;\n+\n+\tstruct reftable_log_record log = {\n+\t\t.refname = \"refname\",\n+\t\t.value_type = REFTABLE_LOG_UPDATE,\n+\t\t.value = {\n+\t\t\t.update = {\n+\t\t\t\t.new_hash = hash1,\n+\t\t\t\t.old_hash = hash2,\n+\t\t\t\t.name = \"My Name\",\n+\t\t\t\t.email = \"myname@invalid\",\n+\t\t\t\t.message = message,\n+\t\t\t},\n+\t\t},\n+\t};\n+\n+\tfor (i = 0; i < sizeof(message)-1; i++)\n+\t\tmessage[i] = (uint8_t)(rand() % 64 + ' ');\n+\n+\treftable_writer_set_limits(w, 1, 1);\n+\n+\terr = reftable_writer_add_log(w, &log);\n+\tEXPECT_ERR(err);\n+\n+\tn = reftable_writer_close(w);\n+\tEXPECT(n == 0);\n+\n+\tstats = writer_stats(w);\n+\tEXPECT(stats->log_stats.blocks > 0);\n+\treftable_writer_free(w);\n+\tw = NULL;\n+\n+\t/* corrupt the data. */\n+\tbuf.buf[50] ^= 0x99;\n+\n+\tblock_source_from_strbuf(&source, &buf);\n+\n+\terr = init_reader(&rd, &source, \"file.log\");\n+\tEXPECT_ERR(err);\n+\n+\terr = reftable_reader_seek_log(&rd, &it, \"refname\");\n+\tEXPECT(err == REFTABLE_ZLIB_ERROR);\n+\n+\treftable_iterator_destroy(&it);\n+\n+\t/* cleanup. */\n+\tstrbuf_release(&buf);\n+\treader_close(&rd);\n+}\n+\n static void test_table_read_write_sequential(void)\n {\n \tchar **names;\n@@ -633,6 +698,7 @@ static void test_corrupt_table(void)\n \n int readwrite_test_main(int argc, const char *argv[])\n {\n+\tRUN_TEST(test_log_zlib_corruption);\n \tRUN_TEST(test_corrupt_table);\n \tRUN_TEST(test_corrupt_table_empty);\n \tRUN_TEST(test_log_write_read);\n-- \ngitgitgadget\n\n"},{"id":"444089","messageId":"51b4a17a2e1f867c47d0b00f72721ef6e21fab04.1639482477.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v4.git.git.1639482476.gitgitgadget@gmail.com","subject":"[PATCH v4 04/11] reftable: check reftable_stack_auto_compact() return value","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-14T11:47:49Z","receivedAt":"2021-12-14T11:48:14Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nFixes a problem detected by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack_test.c | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/reftable/stack_test.c b/reftable/stack_test.c\nindex eb0b7228b0c..d628420e63a 100644\n--- a/reftable/stack_test.c\n+++ b/reftable/stack_test.c\n@@ -814,6 +814,7 @@ static void test_reftable_stack_auto_compaction(void)\n \t\tEXPECT_ERR(err);\n \n \t\terr = reftable_stack_auto_compact(st);\n+\t\tEXPECT_ERR(err);\n \t\tEXPECT(i < 3 || st->merged->stack_len < 2 * fastlog2(i));\n \t}\n \n-- \ngitgitgadget\n\n"},{"id":"444090","messageId":"7a914f77756bc7c9ec5338584f955cb5930ea4e4.1639482477.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v4.git.git.1639482476.gitgitgadget@gmail.com","subject":"[PATCH v4 03/11] reftable: fix resource leak blocksource.c","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-14T11:47:48Z","receivedAt":"2021-12-14T11:48:17Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis would be triggered in the unlikely event of fstat() failing on an opened\nfile.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/blocksource.c | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/reftable/blocksource.c b/reftable/blocksource.c\nindex 0044eecd9aa..2605371c28d 100644\n--- a/reftable/blocksource.c\n+++ b/reftable/blocksource.c\n@@ -134,8 +134,10 @@ int reftable_block_source_from_file(struct reftable_block_source *bs,\n \t}\n \n \terr = fstat(fd, &st);\n-\tif (err < 0)\n-\t\treturn -1;\n+\tif (err < 0) {\n+\t\tclose(fd);\n+\t\treturn REFTABLE_IO_ERROR;\n+\t}\n \n \tp = reftable_calloc(sizeof(struct file_block_source));\n \tp->size = st.st_size;\n-- \ngitgitgadget\n\n"},{"id":"444091","messageId":"43989afcb5a533c892739cf3d95f18b6d447e0e3.1639482477.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v4.git.git.1639482476.gitgitgadget@gmail.com","subject":"[PATCH v4 05/11] reftable: ignore remove() return value in stack_test.c","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-14T11:47:50Z","receivedAt":"2021-12-14T11:48:18Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nIf the cleanup fails, there is nothing we can do.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack_test.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/reftable/stack_test.c b/reftable/stack_test.c\nindex d628420e63a..4b7292945c3 100644\n--- a/reftable/stack_test.c\n+++ b/reftable/stack_test.c\n@@ -89,7 +89,7 @@ static void test_read_file(void)\n \t\tEXPECT(0 == strcmp(want[i], names[i]));\n \t}\n \tfree_names(names);\n-\tremove(fn);\n+\t(void) remove(fn);\n }\n \n static void test_parse_names(void)\n-- \ngitgitgadget\n\n"},{"id":"444092","messageId":"1c7f15d811c4a73f1fd7b24de52fcd1869126982.1639482477.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v4.git.git.1639482476.gitgitgadget@gmail.com","subject":"[PATCH v4 06/11] reftable: fix resource warning","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-14T11:47:51Z","receivedAt":"2021-12-14T11:48:21Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis would trigger in the unlikely event that we are compacting, and the next\navailable file handle is 0.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack.c | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/reftable/stack.c b/reftable/stack.c\nindex df5021ebf08..10dfd370e8e 100644\n--- a/reftable/stack.c\n+++ b/reftable/stack.c\n@@ -877,7 +877,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \tstruct strbuf new_table_path = STRBUF_INIT;\n \tint err = 0;\n \tint have_lock = 0;\n-\tint lock_file_fd = 0;\n+\tint lock_file_fd = -1;\n \tint compact_count = last - first + 1;\n \tchar **listp = NULL;\n \tchar **delete_on_success =\n@@ -911,7 +911,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \t}\n \t/* Don't want to write to the lock for now. */\n \tclose(lock_file_fd);\n-\tlock_file_fd = 0;\n+\tlock_file_fd = -1;\n \n \thave_lock = 1;\n \terr = stack_uptodate(st);\n@@ -932,7 +932,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \n \t\tsublock_file_fd = open(subtab_lock.buf,\n \t\t\t\t O_EXCL | O_CREAT | O_WRONLY, 0644);\n-\t\tif (sublock_file_fd > 0) {\n+\t\tif (sublock_file_fd >= 0) {\n \t\t\tclose(sublock_file_fd);\n \t\t} else if (sublock_file_fd < 0) {\n \t\t\tif (errno == EEXIST) {\n@@ -1013,7 +1013,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \t\tgoto done;\n \t}\n \terr = close(lock_file_fd);\n-\tlock_file_fd = 0;\n+\tlock_file_fd = -1;\n \tif (err < 0) {\n \t\terr = REFTABLE_IO_ERROR;\n \t\tunlink(new_table_path.buf);\n@@ -1050,9 +1050,9 @@ done:\n \t\tlistp++;\n \t}\n \tfree_names(subtable_locks);\n-\tif (lock_file_fd > 0) {\n+\tif (lock_file_fd >= 0) {\n \t\tclose(lock_file_fd);\n-\t\tlock_file_fd = 0;\n+\t\tlock_file_fd = -1;\n \t}\n \tif (have_lock) {\n \t\tunlink(lock_file_name.buf);\n-- \ngitgitgadget\n\n"},{"id":"444093","messageId":"47ba5ddceb8cc864cdc3006f7718e00f6ea2c7fc.1639482477.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v4.git.git.1639482476.gitgitgadget@gmail.com","subject":"[PATCH v4 07/11] reftable: all xxx_free() functions accept NULL arguments","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-14T11:47:52Z","receivedAt":"2021-12-14T11:48:22Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis fixes NULL derefs in error paths. Spotted by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/reader.c | 2 ++\n reftable/writer.c | 2 ++\n 2 files changed, 4 insertions(+)\n\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 006709a645a..733509606a9 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -641,6 +641,8 @@ int reftable_new_reader(struct reftable_reader **p,\n \n void reftable_reader_free(struct reftable_reader *r)\n {\n+\tif (!r)\n+\t\treturn;\n \treader_close(r);\n \treftable_free(r);\n }\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex 3ca721e9f64..9fd24fa93ce 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -150,6 +150,8 @@ void reftable_writer_set_limits(struct reftable_writer *w, uint64_t min,\n \n void reftable_writer_free(struct reftable_writer *w)\n {\n+\tif (!w)\n+\t\treturn;\n \treftable_free(w->block);\n \treftable_free(w);\n }\n-- \ngitgitgadget\n\n"},{"id":"444094","messageId":"aba8b8113ad100156a60c3334d8e867f28f97420.1639482477.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v4.git.git.1639482476.gitgitgadget@gmail.com","subject":"[PATCH v4 08/11] reftable: order unittests by complexity","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-14T11:47:53Z","receivedAt":"2021-12-14T11:48:23Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis is a more practical ordering when working on refactorings of the reftable\ncode.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n t/helper/test-reftable.c | 9 +++++----\n 1 file changed, 5 insertions(+), 4 deletions(-)\n\ndiff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c\nindex 26b03d7b789..1f0a28cbb64 100644\n--- a/t/helper/test-reftable.c\n+++ b/t/helper/test-reftable.c\n@@ -3,15 +3,16 @@\n \n int cmd__reftable(int argc, const char **argv)\n {\n+\t/* test from simple to complex. */\n \tbasics_test_main(argc, argv);\n+\trecord_test_main(argc, argv);\n \tblock_test_main(argc, argv);\n-\tmerged_test_main(argc, argv);\n+\ttree_test_main(argc, argv);\n \tpq_test_main(argc, argv);\n-\trecord_test_main(argc, argv);\n-\trefname_test_main(argc, argv);\n \treadwrite_test_main(argc, argv);\n+\tmerged_test_main(argc, argv);\n \tstack_test_main(argc, argv);\n-\ttree_test_main(argc, argv);\n+\trefname_test_main(argc, argv);\n \treturn 0;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"444095","messageId":"f2af404d7f16a9c6691beabe29113cc5f1c3770c.1639482477.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v4.git.git.1639482476.gitgitgadget@gmail.com","subject":"[PATCH v4 09/11] reftable: drop stray printf in readwrite_test","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-14T11:47:54Z","receivedAt":"2021-12-14T11:48:27Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/readwrite_test.c | 1 -\n 1 file changed, 1 deletion(-)\n\ndiff --git a/reftable/readwrite_test.c b/reftable/readwrite_test.c\nindex 6e88182a83a..9b89a5da103 100644\n--- a/reftable/readwrite_test.c\n+++ b/reftable/readwrite_test.c\n@@ -662,7 +662,6 @@ static void test_write_key_order(void)\n \terr = reftable_writer_add_ref(w, &refs[0]);\n \tEXPECT_ERR(err);\n \terr = reftable_writer_add_ref(w, &refs[1]);\n-\tprintf(\"%d\\n\", err);\n \tEXPECT(err == REFTABLE_API_ERROR);\n \treftable_writer_close(w);\n \treftable_writer_free(w);\n-- \ngitgitgadget\n\n"},{"id":"444096","messageId":"d2ce1bfef9f3f10ba681b72f9be6579bfc8a9491.1639482477.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v4.git.git.1639482476.gitgitgadget@gmail.com","subject":"[PATCH v4 10/11] reftable: handle null refnames in reftable_ref_record_equal","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-14T11:47:55Z","receivedAt":"2021-12-14T11:48:29Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSpotted by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 8536bd03aa9..8bbcbff1e69 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -1154,9 +1154,11 @@ int reftable_ref_record_equal(struct reftable_ref_record *a,\n \t\t\t struct reftable_ref_record *b, int hash_size)\n {\n \tassert(hash_size > 0);\n-\tif (!(0 == strcmp(a->refname, b->refname) &&\n-\t a->update_index == b->update_index &&\n-\t a->value_type == b->value_type))\n+\tif (!null_streq(a->refname, b->refname))\n+\t\treturn 0;\n+\n+\tif (a->update_index != b->update_index ||\n+\t a->value_type != b->value_type)\n \t\treturn 0;\n \n \tswitch (a->value_type) {\n-- \ngitgitgadget\n\n"},{"id":"444097","messageId":"0ffc1cf0e429510d81ad91c6d052e0bd38264f28.1639482477.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v4.git.git.1639482476.gitgitgadget@gmail.com","subject":"[PATCH v4 11/11] reftable: make reftable_record a tagged union","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-14T11:47:56Z","receivedAt":"2021-12-14T11:48:30Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis reduces the amount of glue code, because we don't need a void pointer or\nvtable within the structure.\n\nThe only snag is that reftable_index_record contain a strbuf, so it cannot be\nzero-initialized. To address this, use reftable_new_record() to return fresh\ninstance, given a record type. Since reftable_new_record() doesn't cause heap\nallocation anymore, it should be balanced with reftable_record_release() rather\nthan reftable_record_destroy().\n\nThanks to Peff for the suggestion.\n\nHelped-by: Jeff King \nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/block.c | 2 +-\n reftable/block_test.c | 22 ++---\n reftable/generic.c | 35 ++++----\n reftable/iter.c | 4 +-\n reftable/merged.c | 33 ++++----\n reftable/pq.c | 3 +-\n reftable/pq_test.c | 31 +++----\n reftable/reader.c | 93 +++++++++++----------\n reftable/record.c | 173 ++++++++++++++++-----------------------\n reftable/record.h | 42 ++++------\n reftable/record_test.c | 181 +++++++++++++++++++++--------------------\n reftable/writer.c | 40 +++++----\n 12 files changed, 317 insertions(+), 342 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 6c8e8705205..2170748c5e9 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -421,7 +421,7 @@ int block_reader_seek(struct block_reader *br, struct block_iter *it,\n done:\n \tstrbuf_release(&key);\n \tstrbuf_release(&next.last_key);\n-\treftable_record_destroy(&rec);\n+\treftable_record_release(&rec);\n \n \treturn err;\n }\ndiff --git a/reftable/block_test.c b/reftable/block_test.c\nindex 4b3ea262dcb..fa2ee092ec0 100644\n--- a/reftable/block_test.c\n+++ b/reftable/block_test.c\n@@ -26,8 +26,9 @@ static void test_block_read_write(void)\n \tstruct block_writer bw = {\n \t\t.last_key = STRBUF_INIT,\n \t};\n-\tstruct reftable_ref_record ref = { NULL };\n-\tstruct reftable_record rec = { NULL };\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t};\n \tint i = 0;\n \tint n;\n \tstruct block_reader br = { 0 };\n@@ -40,7 +41,6 @@ static void test_block_read_write(void)\n \tblock.source = malloc_block_source();\n \tblock_writer_init(&bw, BLOCK_TYPE_REF, block.data, block_size,\n \t\t\t header_off, hash_size(GIT_SHA1_FORMAT_ID));\n-\treftable_record_from_ref(&rec, &ref);\n \n \tfor (i = 0; i < N; i++) {\n \t\tchar name[100];\n@@ -48,14 +48,14 @@ static void test_block_read_write(void)\n \t\tsnprintf(name, sizeof(name), \"branch%02d\", i);\n \t\tmemset(hash, i, sizeof(hash));\n \n-\t\tref.refname = name;\n-\t\tref.value_type = REFTABLE_REF_VAL1;\n-\t\tref.value.val1 = hash;\n+\t\trec.u.ref.refname = name;\n+\t\trec.u.ref.value_type = REFTABLE_REF_VAL1;\n+\t\trec.u.ref.value.val1 = hash;\n \n \t\tnames[i] = xstrdup(name);\n \t\tn = block_writer_add(&bw, &rec);\n-\t\tref.refname = NULL;\n-\t\tref.value_type = REFTABLE_REF_DELETION;\n+\t\trec.u.ref.refname = NULL;\n+\t\trec.u.ref.value_type = REFTABLE_REF_DELETION;\n \t\tEXPECT(n == 0);\n \t}\n \n@@ -74,7 +74,7 @@ static void test_block_read_write(void)\n \t\tif (r > 0) {\n \t\t\tbreak;\n \t\t}\n-\t\tEXPECT_STREQ(names[j], ref.refname);\n+\t\tEXPECT_STREQ(names[j], rec.u.ref.refname);\n \t\tj++;\n \t}\n \n@@ -92,7 +92,7 @@ static void test_block_read_write(void)\n \t\tn = block_iter_next(&it, &rec);\n \t\tEXPECT(n == 0);\n \n-\t\tEXPECT_STREQ(names[i], ref.refname);\n+\t\tEXPECT_STREQ(names[i], rec.u.ref.refname);\n \n \t\twant.len--;\n \t\tn = block_reader_seek(&br, &it, &want);\n@@ -100,7 +100,7 @@ static void test_block_read_write(void)\n \n \t\tn = block_iter_next(&it, &rec);\n \t\tEXPECT(n == 0);\n-\t\tEXPECT_STREQ(names[10 * (i / 10)], ref.refname);\n+\t\tEXPECT_STREQ(names[10 * (i / 10)], rec.u.ref.refname);\n \n \t\tblock_iter_close(&it);\n \t}\ndiff --git a/reftable/generic.c b/reftable/generic.c\nindex 7a8a738d860..4446b8ed360 100644\n--- a/reftable/generic.c\n+++ b/reftable/generic.c\n@@ -7,6 +7,7 @@ https://developers.google.com/open-source/licenses/bsd\n */\n \n #include \"basics.h\"\n+#include \"constants.h\"\n #include \"record.h\"\n #include \"generic.h\"\n #include \"reftable-iterator.h\"\n@@ -15,23 +16,21 @@ https://developers.google.com/open-source/licenses/bsd\n int reftable_table_seek_ref(struct reftable_table *tab,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_REF,\n+\t\t\t\t .u.ref = {\n+\t\t\t\t\t .refname = (char *)name,\n+\t\t\t\t } };\n \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n }\n \n int reftable_table_seek_log(struct reftable_table *tab,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = ~((uint64_t)0),\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n+\t\t\t\t .u.log = {\n+\t\t\t\t\t .refname = (char *)name,\n+\t\t\t\t\t .update_index = ~((uint64_t)0),\n+\t\t\t\t } };\n \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n }\n \n@@ -129,17 +128,19 @@ void reftable_iterator_destroy(struct reftable_iterator *it)\n int reftable_iterator_next_ref(struct reftable_iterator *it,\n \t\t\t struct reftable_ref_record *ref)\n {\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, ref);\n-\treturn iterator_next(it, &rec);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_REF };\n+\tint err = iterator_next(it, &rec);\n+\t*ref = rec.u.ref;\n+\treturn err;\n }\n \n int reftable_iterator_next_log(struct reftable_iterator *it,\n \t\t\t struct reftable_log_record *log)\n {\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, log);\n-\treturn iterator_next(it, &rec);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG };\n+\tint err = iterator_next(it, &rec);\n+\t*log = rec.u.log;\n+\treturn err;\n }\n \n int iterator_next(struct reftable_iterator *it, struct reftable_record *rec)\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex 93d04f735b8..a8d174c0406 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -32,7 +32,7 @@ static int filtering_ref_iterator_next(void *iter_arg,\n \t\t\t\t struct reftable_record *rec)\n {\n \tstruct filtering_ref_iterator *fri = iter_arg;\n-\tstruct reftable_ref_record *ref = rec->data;\n+\tstruct reftable_ref_record *ref = &rec->u.ref;\n \tint err = 0;\n \twhile (1) {\n \t\terr = reftable_iterator_next_ref(&fri->it, ref);\n@@ -127,7 +127,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n static int indexed_table_ref_iter_next(void *p, struct reftable_record *rec)\n {\n \tstruct indexed_table_ref_iter *it = p;\n-\tstruct reftable_ref_record *ref = rec->data;\n+\tstruct reftable_ref_record *ref = &rec->u.ref;\n \n \twhile (1) {\n \t\tint err = block_iter_next(&it->cur, rec);\ndiff --git a/reftable/merged.c b/reftable/merged.c\nindex e5b53da6db3..a7c0df91ff0 100644\n--- a/reftable/merged.c\n+++ b/reftable/merged.c\n@@ -30,7 +30,7 @@ static int merged_iter_init(struct merged_iter *mi)\n \n \t\tif (err > 0) {\n \t\t\treftable_iterator_destroy(&mi->stack[i]);\n-\t\t\treftable_record_destroy(&rec);\n+\t\t\treftable_record_release(&rec);\n \t\t} else {\n \t\t\tstruct pq_entry e = {\n \t\t\t\t.rec = rec,\n@@ -57,18 +57,17 @@ static void merged_iter_close(void *p)\n static int merged_iter_advance_nonnull_subiter(struct merged_iter *mi,\n \t\t\t\t\t size_t idx)\n {\n-\tstruct reftable_record rec = reftable_new_record(mi->typ);\n \tstruct pq_entry e = {\n-\t\t.rec = rec,\n+\t\t.rec = reftable_new_record(mi->typ),\n \t\t.index = idx,\n \t};\n-\tint err = iterator_next(&mi->stack[idx], &rec);\n+\tint err = iterator_next(&mi->stack[idx], &e.rec);\n \tif (err < 0)\n \t\treturn err;\n \n \tif (err > 0) {\n \t\treftable_iterator_destroy(&mi->stack[idx]);\n-\t\treftable_record_destroy(&rec);\n+\t\treftable_record_release(&e.rec);\n \t\treturn 0;\n \t}\n \n@@ -126,11 +125,11 @@ static int merged_iter_next_entry(struct merged_iter *mi,\n \t\tif (err < 0) {\n \t\t\treturn err;\n \t\t}\n-\t\treftable_record_destroy(&top.rec);\n+\t\treftable_record_release(&top.rec);\n \t}\n \n \treftable_record_copy_from(rec, &entry.rec, hash_size(mi->hash_id));\n-\treftable_record_destroy(&entry.rec);\n+\treftable_record_release(&entry.rec);\n \tstrbuf_release(&entry_key);\n \treturn 0;\n }\n@@ -290,11 +289,12 @@ int reftable_merged_table_seek_ref(struct reftable_merged_table *mt,\n \t\t\t\t struct reftable_iterator *it,\n \t\t\t\t const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = {\n+\t\t\t.refname = (char *)name,\n+\t\t},\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n \treturn merged_table_seek_record(mt, it, &rec);\n }\n \n@@ -302,12 +302,13 @@ int reftable_merged_table_seek_log_at(struct reftable_merged_table *mt,\n \t\t\t\t struct reftable_iterator *it,\n \t\t\t\t const char *name, uint64_t update_index)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = update_index,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.u.log = {\n+\t\t\t.refname = (char *)name,\n+\t\t\t.update_index = update_index,\n+\t\t}\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n \treturn merged_table_seek_record(mt, it, &rec);\n }\n \ndiff --git a/reftable/pq.c b/reftable/pq.c\nindex efc474017a2..96ca6dd37b3 100644\n--- a/reftable/pq.c\n+++ b/reftable/pq.c\n@@ -74,6 +74,7 @@ struct pq_entry merged_iter_pqueue_remove(struct merged_iter_pqueue *pq)\n void merged_iter_pqueue_add(struct merged_iter_pqueue *pq, struct pq_entry e)\n {\n \tint i = 0;\n+\n \tif (pq->len == pq->cap) {\n \t\tpq->cap = 2 * pq->cap + 1;\n \t\tpq->heap = reftable_realloc(pq->heap,\n@@ -98,7 +99,7 @@ void merged_iter_pqueue_release(struct merged_iter_pqueue *pq)\n {\n \tint i = 0;\n \tfor (i = 0; i < pq->len; i++) {\n-\t\treftable_record_destroy(&pq->heap[i].rec);\n+\t\treftable_record_release(&pq->heap[i].rec);\n \t}\n \tFREE_AND_NULL(pq->heap);\n \tpq->len = pq->cap = 0;\ndiff --git a/reftable/pq_test.c b/reftable/pq_test.c\nindex c9bb05e37b7..e0f5c584ada 100644\n--- a/reftable/pq_test.c\n+++ b/reftable/pq_test.c\n@@ -31,7 +31,7 @@ static void test_pq(void)\n \tint N = ARRAY_SIZE(names) - 1;\n \n \tstruct merged_iter_pqueue pq = { NULL };\n-\tconst char *last = NULL;\n+\tchar *last = NULL;\n \n \tint i = 0;\n \tfor (i = 0; i < N; i++) {\n@@ -42,12 +42,14 @@ static void test_pq(void)\n \n \ti = 1;\n \tdo {\n-\t\tstruct reftable_record rec =\n-\t\t\treftable_new_record(BLOCK_TYPE_REF);\n-\t\tstruct pq_entry e = { 0 };\n-\n-\t\treftable_record_as_ref(&rec)->refname = names[i];\n-\t\te.rec = rec;\n+\t\tstruct pq_entry e = {\n+\t\t\t.rec = {\n+\t\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t\t.u.ref = {\n+\t\t\t\t\t.refname = names[i],\n+\t\t\t\t}\n+\t\t\t}\n+\t\t};\n \t\tmerged_iter_pqueue_add(&pq, e);\n \t\tmerged_iter_pqueue_check(pq);\n \t\ti = (i * 7) % N;\n@@ -55,19 +57,18 @@ static void test_pq(void)\n \n \twhile (!merged_iter_pqueue_is_empty(pq)) {\n \t\tstruct pq_entry e = merged_iter_pqueue_remove(&pq);\n-\t\tstruct reftable_ref_record *ref =\n-\t\t\treftable_record_as_ref(&e.rec);\n-\n+\t\tstruct reftable_record *rec = &e.rec;\n \t\tmerged_iter_pqueue_check(pq);\n \n+\t\tEXPECT(reftable_record_type(rec) == BLOCK_TYPE_REF);\n \t\tif (last) {\n-\t\t\tEXPECT(strcmp(last, ref->refname) < 0);\n+\t\t\tEXPECT(strcmp(last, rec->u.ref.refname) < 0);\n \t\t}\n-\t\tlast = ref->refname;\n-\t\tref->refname = NULL;\n-\t\treftable_free(ref);\n+\t\t// this is names[i], so don't dealloc.\n+\t\tlast = rec->u.ref.refname;\n+\t\trec->u.ref.refname = NULL;\n+\t\treftable_record_release(rec);\n \t}\n-\n \tfor (i = 0; i < N; i++) {\n \t\treftable_free(names[i]);\n \t}\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 733509606a9..0790f654366 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -239,8 +239,7 @@ static int table_iter_next_in_block(struct table_iter *ti,\n {\n \tint res = block_iter_next(&ti->bi, rec);\n \tif (res == 0 && reftable_record_type(rec) == BLOCK_TYPE_REF) {\n-\t\t((struct reftable_ref_record *)rec->data)->update_index +=\n-\t\t\tti->r->min_update_index;\n+\t\trec->u.ref.update_index += ti->r->min_update_index;\n \t}\n \n \treturn res;\n@@ -437,8 +436,7 @@ static int reader_start(struct reftable_reader *r, struct table_iter *ti,\n static int reader_seek_linear(struct reftable_reader *r, struct table_iter *ti,\n \t\t\t struct reftable_record *want)\n {\n-\tstruct reftable_record rec =\n-\t\treftable_new_record(reftable_record_type(want));\n+\tstruct reftable_record rec = reftable_new_record(reftable_record_type(want));\n \tstruct strbuf want_key = STRBUF_INIT;\n \tstruct strbuf got_key = STRBUF_INIT;\n \tstruct table_iter next = TABLE_ITER_INIT;\n@@ -475,7 +473,7 @@ static int reader_seek_linear(struct reftable_reader *r, struct table_iter *ti,\n \n done:\n \tblock_iter_close(&next.bi);\n-\treftable_record_destroy(&rec);\n+\treftable_record_release(&rec);\n \tstrbuf_release(&want_key);\n \tstrbuf_release(&got_key);\n \treturn err;\n@@ -485,34 +483,36 @@ static int reader_seek_indexed(struct reftable_reader *r,\n \t\t\t struct reftable_iterator *it,\n \t\t\t struct reftable_record *rec)\n {\n-\tstruct reftable_index_record want_index = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record want_index_rec = { NULL };\n-\tstruct reftable_index_record index_result = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record index_result_rec = { NULL };\n+\tstruct reftable_record want_index = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = { .last_key = STRBUF_INIT }\n+\t};\n+\tstruct reftable_record index_result = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = { .last_key = STRBUF_INIT },\n+\t};\n \tstruct table_iter index_iter = TABLE_ITER_INIT;\n \tstruct table_iter next = TABLE_ITER_INIT;\n \tint err = 0;\n \n-\treftable_record_key(rec, &want_index.last_key);\n-\treftable_record_from_index(&want_index_rec, &want_index);\n-\treftable_record_from_index(&index_result_rec, &index_result);\n-\n+\treftable_record_key(rec, &want_index.u.idx.last_key);\n \terr = reader_start(r, &index_iter, reftable_record_type(rec), 1);\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = reader_seek_linear(r, &index_iter, &want_index_rec);\n+\terr = reader_seek_linear(r, &index_iter, &want_index);\n \twhile (1) {\n-\t\terr = table_iter_next(&index_iter, &index_result_rec);\n+\t\terr = table_iter_next(&index_iter, &index_result);\n \t\ttable_iter_block_done(&index_iter);\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = reader_table_iter_at(r, &next, index_result.offset, 0);\n+\t\terr = reader_table_iter_at(r, &next, index_result.u.idx.offset,\n+\t\t\t\t\t 0);\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = block_iter_seek(&next.bi, &want_index.last_key);\n+\t\terr = block_iter_seek(&next.bi, &want_index.u.idx.last_key);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n@@ -540,8 +540,8 @@ static int reader_seek_indexed(struct reftable_reader *r,\n done:\n \tblock_iter_close(&next.bi);\n \ttable_iter_close(&index_iter);\n-\treftable_record_release(&want_index_rec);\n-\treftable_record_release(&index_result_rec);\n+\treftable_record_release(&want_index);\n+\treftable_record_release(&index_result);\n \treturn err;\n }\n \n@@ -549,8 +549,7 @@ static int reader_seek_internal(struct reftable_reader *r,\n \t\t\t\tstruct reftable_iterator *it,\n \t\t\t\tstruct reftable_record *rec)\n {\n-\tstruct reftable_reader_offsets *offs =\n-\t\treader_offsets_for(r, reftable_record_type(rec));\n+\tstruct reftable_reader_offsets *offs = reader_offsets_for(r, reftable_record_type(rec));\n \tuint64_t idx = offs->index_offset;\n \tstruct table_iter ti = TABLE_ITER_INIT;\n \tint err = 0;\n@@ -590,11 +589,12 @@ static int reader_seek(struct reftable_reader *r, struct reftable_iterator *it,\n int reftable_reader_seek_ref(struct reftable_reader *r,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = {\n+\t\t\t.refname = (char *)name,\n+\t\t},\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n \treturn reader_seek(r, it, &rec);\n }\n \n@@ -602,12 +602,13 @@ int reftable_reader_seek_log_at(struct reftable_reader *r,\n \t\t\t\tstruct reftable_iterator *it, const char *name,\n \t\t\t\tuint64_t update_index)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = update_index,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.u.log = {\n+\t\t\t.refname = (char *)name,\n+\t\t\t.update_index = update_index,\n+\t\t}\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n \treturn reader_seek(r, it, &rec);\n }\n \n@@ -651,31 +652,33 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \t\t\t\t\t struct reftable_iterator *it,\n \t\t\t\t\t uint8_t *oid)\n {\n-\tstruct reftable_obj_record want = {\n-\t\t.hash_prefix = oid,\n-\t\t.hash_prefix_len = r->object_id_len,\n+\tstruct reftable_record want = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.u.obj = {\n+\t\t\t.hash_prefix = oid,\n+\t\t\t.hash_prefix_len = r->object_id_len,\n+\t\t},\n \t};\n-\tstruct reftable_record want_rec = { NULL };\n \tstruct reftable_iterator oit = { NULL };\n-\tstruct reftable_obj_record got = { NULL };\n-\tstruct reftable_record got_rec = { NULL };\n+\tstruct reftable_record got = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.u.obj = { 0 },\n+\t};\n \tint err = 0;\n \tstruct indexed_table_ref_iter *itr = NULL;\n \n \t/* Look through the reverse index. */\n-\treftable_record_from_obj(&want_rec, &want);\n-\terr = reader_seek(r, &oit, &want_rec);\n+\terr = reader_seek(r, &oit, &want);\n \tif (err != 0)\n \t\tgoto done;\n \n \t/* read out the reftable_obj_record */\n-\treftable_record_from_obj(&got_rec, &got);\n-\terr = iterator_next(&oit, &got_rec);\n+\terr = iterator_next(&oit, &got);\n \tif (err < 0)\n \t\tgoto done;\n \n-\tif (err > 0 ||\n-\t memcmp(want.hash_prefix, got.hash_prefix, r->object_id_len)) {\n+\tif (err > 0 || memcmp(want.u.obj.hash_prefix, got.u.obj.hash_prefix,\n+\t\t\t r->object_id_len)) {\n \t\t/* didn't find it; return empty iterator */\n \t\titerator_set_empty(it);\n \t\terr = 0;\n@@ -683,15 +686,15 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \t}\n \n \terr = new_indexed_table_ref_iter(&itr, r, oid, hash_size(r->hash_id),\n-\t\t\t\t\t got.offsets, got.offset_len);\n+\t\t\t\t\t got.u.obj.offsets, got.u.obj.offset_len);\n \tif (err < 0)\n \t\tgoto done;\n-\tgot.offsets = NULL;\n+\tgot.u.obj.offsets = NULL;\n \titerator_from_indexed_table_ref_iter(it, itr);\n \n done:\n \treftable_iterator_destroy(&oit);\n-\treftable_record_release(&got_rec);\n+\treftable_record_release(&got);\n \treturn err;\n }\n \ndiff --git a/reftable/record.c b/reftable/record.c\nindex 8bbcbff1e69..f766ee20ace 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -15,6 +15,10 @@ https://developers.google.com/open-source/licenses/bsd\n #include \"reftable-error.h\"\n #include \"basics.h\"\n \n+static struct reftable_record_vtable *\n+reftable_record_vtable(struct reftable_record *rec);\n+static void *reftable_record_data(struct reftable_record *rec);\n+\n int get_var_int(uint64_t *dest, struct string_view *in)\n {\n \tint ptr = 0;\n@@ -926,58 +930,6 @@ static struct reftable_record_vtable reftable_log_record_vtable = {\n \t.is_deletion = &reftable_log_record_is_deletion_void,\n };\n \n-struct reftable_record reftable_new_record(uint8_t typ)\n-{\n-\tstruct reftable_record rec = { NULL };\n-\tswitch (typ) {\n-\tcase BLOCK_TYPE_REF: {\n-\t\tstruct reftable_ref_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_ref_record));\n-\t\treftable_record_from_ref(&rec, r);\n-\t\treturn rec;\n-\t}\n-\n-\tcase BLOCK_TYPE_OBJ: {\n-\t\tstruct reftable_obj_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_obj_record));\n-\t\treftable_record_from_obj(&rec, r);\n-\t\treturn rec;\n-\t}\n-\tcase BLOCK_TYPE_LOG: {\n-\t\tstruct reftable_log_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_log_record));\n-\t\treftable_record_from_log(&rec, r);\n-\t\treturn rec;\n-\t}\n-\tcase BLOCK_TYPE_INDEX: {\n-\t\tstruct reftable_index_record empty = { .last_key =\n-\t\t\t\t\t\t\t STRBUF_INIT };\n-\t\tstruct reftable_index_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_index_record));\n-\t\t*r = empty;\n-\t\treftable_record_from_index(&rec, r);\n-\t\treturn rec;\n-\t}\n-\t}\n-\tabort();\n-\treturn rec;\n-}\n-\n-/* clear out the record, yielding the reftable_record data that was\n- * encapsulated. */\n-static void *reftable_record_yield(struct reftable_record *rec)\n-{\n-\tvoid *p = rec->data;\n-\trec->data = NULL;\n-\treturn p;\n-}\n-\n-void reftable_record_destroy(struct reftable_record *rec)\n-{\n-\treftable_record_release(rec);\n-\treftable_free(reftable_record_yield(rec));\n-}\n-\n static void reftable_index_record_key(const void *r, struct strbuf *dest)\n {\n \tconst struct reftable_index_record *rec = r;\n@@ -1055,91 +1007,52 @@ static struct reftable_record_vtable reftable_index_record_vtable = {\n \n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)\n {\n-\trec->ops->key(rec->data, dest);\n+\treftable_record_vtable(rec)->key(reftable_record_data(rec), dest);\n }\n \n uint8_t reftable_record_type(struct reftable_record *rec)\n {\n-\treturn rec->ops->type;\n+\treturn rec->type;\n }\n \n int reftable_record_encode(struct reftable_record *rec, struct string_view dest,\n \t\t\t int hash_size)\n {\n-\treturn rec->ops->encode(rec->data, dest, hash_size);\n+\treturn reftable_record_vtable(rec)->encode(reftable_record_data(rec),\n+\t\t\t\t\t\t dest, hash_size);\n }\n \n void reftable_record_copy_from(struct reftable_record *rec,\n \t\t\t struct reftable_record *src, int hash_size)\n {\n-\tassert(src->ops->type == rec->ops->type);\n+\tassert(src->type == rec->type);\n \n-\trec->ops->copy_from(rec->data, src->data, hash_size);\n+\treftable_record_vtable(rec)->copy_from(reftable_record_data(rec),\n+\t\t\t\t\t reftable_record_data(src),\n+\t\t\t\t\t hash_size);\n }\n \n uint8_t reftable_record_val_type(struct reftable_record *rec)\n {\n-\treturn rec->ops->val_type(rec->data);\n+\treturn reftable_record_vtable(rec)->val_type(reftable_record_data(rec));\n }\n \n int reftable_record_decode(struct reftable_record *rec, struct strbuf key,\n \t\t\t uint8_t extra, struct string_view src, int hash_size)\n {\n-\treturn rec->ops->decode(rec->data, key, extra, src, hash_size);\n+\treturn reftable_record_vtable(rec)->decode(reftable_record_data(rec),\n+\t\t\t\t\t\t key, extra, src, hash_size);\n }\n \n void reftable_record_release(struct reftable_record *rec)\n {\n-\trec->ops->release(rec->data);\n+\treftable_record_vtable(rec)->release(reftable_record_data(rec));\n }\n \n int reftable_record_is_deletion(struct reftable_record *rec)\n {\n-\treturn rec->ops->is_deletion(rec->data);\n-}\n-\n-void reftable_record_from_ref(struct reftable_record *rec,\n-\t\t\t struct reftable_ref_record *ref_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = ref_rec;\n-\trec->ops = &reftable_ref_record_vtable;\n-}\n-\n-void reftable_record_from_obj(struct reftable_record *rec,\n-\t\t\t struct reftable_obj_record *obj_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = obj_rec;\n-\trec->ops = &reftable_obj_record_vtable;\n-}\n-\n-void reftable_record_from_index(struct reftable_record *rec,\n-\t\t\t\tstruct reftable_index_record *index_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = index_rec;\n-\trec->ops = &reftable_index_record_vtable;\n-}\n-\n-void reftable_record_from_log(struct reftable_record *rec,\n-\t\t\t struct reftable_log_record *log_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = log_rec;\n-\trec->ops = &reftable_log_record_vtable;\n-}\n-\n-struct reftable_ref_record *reftable_record_as_ref(struct reftable_record *rec)\n-{\n-\tassert(reftable_record_type(rec) == BLOCK_TYPE_REF);\n-\treturn rec->data;\n-}\n-\n-struct reftable_log_record *reftable_record_as_log(struct reftable_record *rec)\n-{\n-\tassert(reftable_record_type(rec) == BLOCK_TYPE_LOG);\n-\treturn rec->data;\n+\treturn reftable_record_vtable(rec)->is_deletion(\n+\t\treftable_record_data(rec));\n }\n \n static int hash_equal(uint8_t *a, uint8_t *b, int hash_size)\n@@ -1212,3 +1125,53 @@ void string_view_consume(struct string_view *s, int n)\n \ts->buf += n;\n \ts->len -= n;\n }\n+\n+static void *reftable_record_data(struct reftable_record *rec)\n+{\n+\tswitch (rec->type) {\n+\tcase BLOCK_TYPE_REF:\n+\t\treturn &rec->u.ref;\n+\tcase BLOCK_TYPE_LOG:\n+\t\treturn &rec->u.log;\n+\tcase BLOCK_TYPE_INDEX:\n+\t\treturn &rec->u.idx;\n+\tcase BLOCK_TYPE_OBJ:\n+\t\treturn &rec->u.obj;\n+\t}\n+\tabort();\n+}\n+\n+static struct reftable_record_vtable *\n+reftable_record_vtable(struct reftable_record *rec)\n+{\n+\tswitch (rec->type) {\n+\tcase BLOCK_TYPE_REF:\n+\t\treturn &reftable_ref_record_vtable;\n+\tcase BLOCK_TYPE_LOG:\n+\t\treturn &reftable_log_record_vtable;\n+\tcase BLOCK_TYPE_INDEX:\n+\t\treturn &reftable_index_record_vtable;\n+\tcase BLOCK_TYPE_OBJ:\n+\t\treturn &reftable_obj_record_vtable;\n+\t}\n+\tabort();\n+}\n+\n+struct reftable_record reftable_new_record(uint8_t typ)\n+{\n+\tstruct reftable_record clean_idx = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = {\n+\t\t\t.last_key = STRBUF_INIT,\n+\t\t},\n+\t};\n+\tstruct reftable_record clean = {\n+\t\t.type = typ,\n+\t};\n+\n+\tif (typ == BLOCK_TYPE_INDEX) {\n+\t\treturn clean_idx;\n+\t}\n+\n+\treturn clean;\n+}\ndiff --git a/reftable/record.h b/reftable/record.h\nindex 498e8c50bf4..407cdaba018 100644\n--- a/reftable/record.h\n+++ b/reftable/record.h\n@@ -60,16 +60,10 @@ struct reftable_record_vtable {\n \tint (*is_deletion)(const void *rec);\n };\n \n-/* record is a generic wrapper for different types of records. */\n-struct reftable_record {\n-\tvoid *data;\n-\tstruct reftable_record_vtable *ops;\n-};\n-\n /* returns true for recognized block types. Block start with the block type. */\n int reftable_is_block_type(uint8_t typ);\n \n-/* creates a malloced record of the given type. Dispose with record_destroy */\n+/* return an initialized record for the given type */\n struct reftable_record reftable_new_record(uint8_t typ);\n \n /* Encode `key` into `dest`. Sets `is_restart` to indicate a restart. Returns\n@@ -97,6 +91,22 @@ struct reftable_obj_record {\n \tint offset_len;\n };\n \n+/* record is a generic wrapper for different types of records. It is normally\n+ * created on the stack, or embedded within another struct. If the type is\n+ * known, a fresh instance can be initialized explicitly. Otherwise, use\n+ * reftable_new_record() to initialize generically (as the index_record is not\n+ * valid as 0-initialized structure)\n+ */\n+struct reftable_record {\n+\tuint8_t type;\n+\tunion {\n+\t\tstruct reftable_ref_record ref;\n+\t\tstruct reftable_log_record log;\n+\t\tstruct reftable_obj_record obj;\n+\t\tstruct reftable_index_record idx;\n+\t} u;\n+};\n+\n /* see struct record_vtable */\n \n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest);\n@@ -111,25 +121,9 @@ int reftable_record_decode(struct reftable_record *rec, struct strbuf key,\n \t\t\t int hash_size);\n int reftable_record_is_deletion(struct reftable_record *rec);\n \n-/* zeroes out the embedded record */\n+/* frees and zeroes out the embedded record */\n void reftable_record_release(struct reftable_record *rec);\n \n-/* clear and deallocate embedded record, and zero `rec`. */\n-void reftable_record_destroy(struct reftable_record *rec);\n-\n-/* initialize generic records from concrete records. The generic record should\n- * be zeroed out. */\n-void reftable_record_from_obj(struct reftable_record *rec,\n-\t\t\t struct reftable_obj_record *objrec);\n-void reftable_record_from_index(struct reftable_record *rec,\n-\t\t\t\tstruct reftable_index_record *idxrec);\n-void reftable_record_from_ref(struct reftable_record *rec,\n-\t\t\t struct reftable_ref_record *refrec);\n-void reftable_record_from_log(struct reftable_record *rec,\n-\t\t\t struct reftable_log_record *logrec);\n-struct reftable_ref_record *reftable_record_as_ref(struct reftable_record *ref);\n-struct reftable_log_record *reftable_record_as_log(struct reftable_record *ref);\n-\n /* for qsort. */\n int reftable_ref_record_compare_name(const void *a, const void *b);\n \ndiff --git a/reftable/record_test.c b/reftable/record_test.c\nindex f4ad7cace41..96f65035c1b 100644\n--- a/reftable/record_test.c\n+++ b/reftable/record_test.c\n@@ -16,24 +16,22 @@\n \n static void test_copy(struct reftable_record *rec)\n {\n-\tstruct reftable_record copy =\n-\t\treftable_new_record(reftable_record_type(rec));\n+\tstruct reftable_record copy = reftable_new_record(reftable_record_type(rec));\n+\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \t/* do it twice to catch memory leaks */\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n-\tswitch (reftable_record_type(©)) {\n+\tswitch (copy.type) {\n \tcase BLOCK_TYPE_REF:\n-\t\tEXPECT(reftable_ref_record_equal(reftable_record_as_ref(©),\n-\t\t\t\t\t\t reftable_record_as_ref(rec),\n+\t\tEXPECT(reftable_ref_record_equal(©.u.ref, &rec->u.ref,\n \t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\tbreak;\n \tcase BLOCK_TYPE_LOG:\n-\t\tEXPECT(reftable_log_record_equal(reftable_record_as_log(©),\n-\t\t\t\t\t\t reftable_record_as_log(rec),\n+\t\tEXPECT(reftable_log_record_equal(©.u.log, &rec->u.log,\n \t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\tbreak;\n \t}\n-\treftable_record_destroy(©);\n+\treftable_record_release(©);\n }\n \n static void test_varint_roundtrip(void)\n@@ -106,61 +104,60 @@ static void test_reftable_ref_record_roundtrip(void)\n \tint i = 0;\n \n \tfor (i = REFTABLE_REF_DELETION; i < REFTABLE_NR_REF_VALUETYPES; i++) {\n-\t\tstruct reftable_ref_record in = { NULL };\n-\t\tstruct reftable_ref_record out = { NULL };\n-\t\tstruct reftable_record rec_out = { NULL };\n+\t\tstruct reftable_record in = {\n+\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t};\n+\t\tstruct reftable_record out = {\n+\t\t\t.type = BLOCK_TYPE_REF\n+\t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n-\t\tstruct reftable_record rec = { NULL };\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n \t\t\t.buf = buffer,\n \t\t\t.len = sizeof(buffer),\n \t\t};\n-\n \t\tint n, m;\n \n-\t\tin.value_type = i;\n+\t\tin.u.ref.value_type = i;\n \t\tswitch (i) {\n \t\tcase REFTABLE_REF_DELETION:\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_VAL1:\n-\t\t\tin.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val1, 1);\n+\t\t\tin.u.ref.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n+\t\t\tset_hash(in.u.ref.value.val1, 1);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_VAL2:\n-\t\t\tin.value.val2.value = reftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val2.value, 1);\n-\t\t\tin.value.val2.target_value =\n+\t\t\tin.u.ref.value.val2.value =\n+\t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n+\t\t\tset_hash(in.u.ref.value.val2.value, 1);\n+\t\t\tin.u.ref.value.val2.target_value =\n \t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val2.target_value, 2);\n+\t\t\tset_hash(in.u.ref.value.val2.target_value, 2);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_SYMREF:\n-\t\t\tin.value.symref = xstrdup(\"target\");\n+\t\t\tin.u.ref.value.symref = xstrdup(\"target\");\n \t\t\tbreak;\n \t\t}\n-\t\tin.refname = xstrdup(\"refs/heads/master\");\n+\t\tin.u.ref.refname = xstrdup(\"refs/heads/master\");\n \n-\t\treftable_record_from_ref(&rec, &in);\n-\t\ttest_copy(&rec);\n+\t\ttest_copy(&in);\n \n-\t\tEXPECT(reftable_record_val_type(&rec) == i);\n+\t\tEXPECT(reftable_record_val_type(&in) == i);\n \n-\t\treftable_record_key(&rec, &key);\n-\t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\t\treftable_record_key(&in, &key);\n+\t\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n > 0);\n \n \t\t/* decode into a non-zero reftable_record to test for leaks. */\n-\n-\t\treftable_record_from_ref(&rec_out, &out);\n-\t\tm = reftable_record_decode(&rec_out, key, i, dest,\n-\t\t\t\t\t GIT_SHA1_RAWSZ);\n+\t\tm = reftable_record_decode(&out, key, i, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_ref_record_equal(&in, &out, GIT_SHA1_RAWSZ));\n-\t\treftable_record_release(&rec_out);\n+\t\tEXPECT(reftable_ref_record_equal(&in.u.ref, &out.u.ref,\n+\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n+\t\treftable_record_release(&in);\n \n \t\tstrbuf_release(&key);\n-\t\treftable_ref_record_release(&in);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n@@ -213,7 +210,9 @@ static void test_reftable_log_record_roundtrip(void)\n \tset_test_hash(in[0].value.update.new_hash, 1);\n \tset_test_hash(in[0].value.update.old_hash, 2);\n \tfor (i = 0; i < ARRAY_SIZE(in); i++) {\n-\t\tstruct reftable_record rec = { NULL };\n+\t\tstruct reftable_record rec = {\n+\t\t\t.type = BLOCK_TYPE_LOG\n+\t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n@@ -221,23 +220,25 @@ static void test_reftable_log_record_roundtrip(void)\n \t\t\t.len = sizeof(buffer),\n \t\t};\n \t\t/* populate out, to check for leaks. */\n-\t\tstruct reftable_log_record out = {\n-\t\t\t.refname = xstrdup(\"old name\"),\n-\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n-\t\t\t.value = {\n-\t\t\t\t.update = {\n-\t\t\t\t\t.new_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n-\t\t\t\t\t.old_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n-\t\t\t\t\t.name = xstrdup(\"old name\"),\n-\t\t\t\t\t.email = xstrdup(\"old@email\"),\n-\t\t\t\t\t.message = xstrdup(\"old message\"),\n+\t\tstruct reftable_record out = {\n+\t\t\t.type = BLOCK_TYPE_LOG,\n+\t\t\t.u.log = {\n+\t\t\t\t.refname = xstrdup(\"old name\"),\n+\t\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n+\t\t\t\t.value = {\n+\t\t\t\t\t.update = {\n+\t\t\t\t\t\t.new_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t\t.old_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t\t.name = xstrdup(\"old name\"),\n+\t\t\t\t\t\t.email = xstrdup(\"old@email\"),\n+\t\t\t\t\t\t.message = xstrdup(\"old message\"),\n+\t\t\t\t\t},\n \t\t\t\t},\n \t\t\t},\n \t\t};\n-\t\tstruct reftable_record rec_out = { NULL };\n \t\tint n, m, valtype;\n \n-\t\treftable_record_from_log(&rec, &in[i]);\n+\t\trec.u.log = in[i];\n \n \t\ttest_copy(&rec);\n \n@@ -245,16 +246,16 @@ static void test_reftable_log_record_roundtrip(void)\n \n \t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n >= 0);\n-\t\treftable_record_from_log(&rec_out, &out);\n \t\tvaltype = reftable_record_val_type(&rec);\n-\t\tm = reftable_record_decode(&rec_out, key, valtype, dest,\n+\t\tm = reftable_record_decode(&out, key, valtype, dest,\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_log_record_equal(&in[i], &out, GIT_SHA1_RAWSZ));\n+\t\tEXPECT(reftable_log_record_equal(&in[i], &out.u.log,\n+\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\treftable_log_record_release(&in[i]);\n \t\tstrbuf_release(&key);\n-\t\treftable_record_release(&rec_out);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n@@ -322,47 +323,52 @@ static void test_reftable_obj_record_roundtrip(void)\n \t\t\t\t\t } };\n \tint i = 0;\n \tfor (i = 0; i < ARRAY_SIZE(recs); i++) {\n-\t\tstruct reftable_obj_record in = recs[i];\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n \t\t\t.buf = buffer,\n \t\t\t.len = sizeof(buffer),\n \t\t};\n-\t\tstruct reftable_record rec = { NULL };\n+\t\tstruct reftable_record in = {\n+\t\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t\t.u.obj = recs[i],\n+\t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n-\t\tstruct reftable_obj_record out = { NULL };\n-\t\tstruct reftable_record rec_out = { NULL };\n+\t\tstruct reftable_record out = {\n+\t\t\t.type = BLOCK_TYPE_OBJ\n+\t\t};\n \t\tint n, m;\n \t\tuint8_t extra;\n \n-\t\treftable_record_from_obj(&rec, &in);\n-\t\ttest_copy(&rec);\n-\t\treftable_record_key(&rec, &key);\n-\t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\t\ttest_copy(&in);\n+\t\treftable_record_key(&in, &key);\n+\t\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n > 0);\n-\t\textra = reftable_record_val_type(&rec);\n-\t\treftable_record_from_obj(&rec_out, &out);\n-\t\tm = reftable_record_decode(&rec_out, key, extra, dest,\n+\t\textra = reftable_record_val_type(&in);\n+\t\tm = reftable_record_decode(&out, key, extra, dest,\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(in.hash_prefix_len == out.hash_prefix_len);\n-\t\tEXPECT(in.offset_len == out.offset_len);\n-\n-\t\tEXPECT(!memcmp(in.hash_prefix, out.hash_prefix,\n-\t\t\t in.hash_prefix_len));\n-\t\tEXPECT(0 == memcmp(in.offsets, out.offsets,\n-\t\t\t\t sizeof(uint64_t) * in.offset_len));\n+\t\tEXPECT(in.u.obj.hash_prefix_len == out.u.obj.hash_prefix_len);\n+\t\tEXPECT(in.u.obj.offset_len == out.u.obj.offset_len);\n+\t\tif (in.u.obj.hash_prefix_len)\n+\t\t\tEXPECT(!memcmp(in.u.obj.hash_prefix, out.u.obj.hash_prefix,\n+\t\t\t\t in.u.obj.hash_prefix_len));\n+\t\tif (in.u.obj.offset_len)\n+\t\t\tEXPECT(!memcmp(in.u.obj.offsets, out.u.obj.offsets,\n+\t\t\t\t\t sizeof(uint64_t) * in.u.obj.offset_len));\n \t\tstrbuf_release(&key);\n-\t\treftable_record_release(&rec_out);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n static void test_reftable_index_record_roundtrip(void)\n {\n-\tstruct reftable_index_record in = {\n-\t\t.offset = 42,\n-\t\t.last_key = STRBUF_INIT,\n+\tstruct reftable_record in = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = {\n+\t\t\t.offset = 42,\n+\t\t\t.last_key = STRBUF_INIT,\n+\t\t},\n \t};\n \tuint8_t buffer[1024] = { 0 };\n \tstruct string_view dest = {\n@@ -370,31 +376,30 @@ static void test_reftable_index_record_roundtrip(void)\n \t\t.len = sizeof(buffer),\n \t};\n \tstruct strbuf key = STRBUF_INIT;\n-\tstruct reftable_record rec = { NULL };\n-\tstruct reftable_index_record out = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record out_rec = { NULL };\n+\tstruct reftable_record out = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = { .last_key = STRBUF_INIT },\n+\t};\n \tint n, m;\n \tuint8_t extra;\n \n-\tstrbuf_addstr(&in.last_key, \"refs/heads/master\");\n-\treftable_record_from_index(&rec, &in);\n-\treftable_record_key(&rec, &key);\n-\ttest_copy(&rec);\n+\tstrbuf_addstr(&in.u.idx.last_key, \"refs/heads/master\");\n+\treftable_record_key(&in, &key);\n+\ttest_copy(&in);\n \n-\tEXPECT(0 == strbuf_cmp(&key, &in.last_key));\n-\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\tEXPECT(0 == strbuf_cmp(&key, &in.u.idx.last_key));\n+\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(n > 0);\n \n-\textra = reftable_record_val_type(&rec);\n-\treftable_record_from_index(&out_rec, &out);\n-\tm = reftable_record_decode(&out_rec, key, extra, dest, GIT_SHA1_RAWSZ);\n+\textra = reftable_record_val_type(&in);\n+\tm = reftable_record_decode(&out, key, extra, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(m == n);\n \n-\tEXPECT(in.offset == out.offset);\n+\tEXPECT(in.u.idx.offset == out.u.idx.offset);\n \n-\treftable_record_release(&out_rec);\n+\treftable_record_release(&out);\n \tstrbuf_release(&key);\n-\tstrbuf_release(&in.last_key);\n+\tstrbuf_release(&in.u.idx.last_key);\n }\n \n int record_test_main(int argc, const char *argv[])\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex 9fd24fa93ce..51066c36a61 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -253,8 +253,10 @@ done:\n int reftable_writer_add_ref(struct reftable_writer *w,\n \t\t\t struct reftable_ref_record *ref)\n {\n-\tstruct reftable_record rec = { NULL };\n-\tstruct reftable_ref_record copy = *ref;\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = *ref,\n+\t};\n \tint err = 0;\n \n \tif (ref->refname == NULL)\n@@ -263,8 +265,7 @@ int reftable_writer_add_ref(struct reftable_writer *w,\n \t ref->update_index > w->max_update_index)\n \t\treturn REFTABLE_API_ERROR;\n \n-\treftable_record_from_ref(&rec, ©);\n-\tcopy.update_index -= w->min_update_index;\n+\trec.u.ref.update_index -= w->min_update_index;\n \n \terr = writer_add_record(w, &rec);\n \tif (err < 0)\n@@ -303,7 +304,10 @@ int reftable_writer_add_refs(struct reftable_writer *w,\n static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n \t\t\t\t\t struct reftable_log_record *log)\n {\n-\tstruct reftable_record rec = { NULL };\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.u.log = *log,\n+\t};\n \tif (w->block_writer &&\n \t block_writer_type(w->block_writer) == BLOCK_TYPE_REF) {\n \t\tint err = writer_finish_public_section(w);\n@@ -313,8 +317,6 @@ static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n \n \tw->next -= w->pending_padding;\n \tw->pending_padding = 0;\n-\n-\treftable_record_from_log(&rec, log);\n \treturn writer_add_record(w, &rec);\n }\n \n@@ -395,8 +397,10 @@ static int writer_finish_section(struct reftable_writer *w)\n \t\tw->index_len = 0;\n \t\tw->index_cap = 0;\n \t\tfor (i = 0; i < idx_len; i++) {\n-\t\t\tstruct reftable_record rec = { NULL };\n-\t\t\treftable_record_from_index(&rec, idx + i);\n+\t\t\tstruct reftable_record rec = {\n+\t\t\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t\t\t.u.idx = idx[i],\n+\t\t\t};\n \t\t\tif (block_writer_add(w->block_writer, &rec) == 0) {\n \t\t\t\tcontinue;\n \t\t\t}\n@@ -464,17 +468,18 @@ static void write_object_record(void *void_arg, void *key)\n {\n \tstruct write_record_arg *arg = void_arg;\n \tstruct obj_index_tree_node *entry = key;\n-\tstruct reftable_obj_record obj_rec = {\n-\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n-\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n-\t\t.offsets = entry->offsets,\n-\t\t.offset_len = entry->offset_len,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.u.obj = {\n+\t\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n+\t\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n+\t\t\t.offsets = entry->offsets,\n+\t\t\t.offset_len = entry->offset_len,\n+\t\t}\n \t};\n-\tstruct reftable_record rec = { NULL };\n \tif (arg->err < 0)\n \t\tgoto done;\n \n-\treftable_record_from_obj(&rec, &obj_rec);\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \tif (arg->err == 0)\n \t\tgoto done;\n@@ -487,7 +492,8 @@ static void write_object_record(void *void_arg, void *key)\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \tif (arg->err == 0)\n \t\tgoto done;\n-\tobj_rec.offset_len = 0;\n+\n+\trec.u.obj.offset_len = 0;\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \n \t/* Should be able to write into a fresh block. */\n-- \ngitgitgadget\n"},{"id":"444795","messageId":"e7f1be7bbec0427d5c10453631c3486936ed2bf0.1640199396.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","subject":"[PATCH v5 01/16] reftable: fix OOB stack write in print functions","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-22T18:56:21Z","receivedAt":"2021-12-22T18:56:40Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 6a5dac32dc6..8536bd03aa9 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -254,7 +254,7 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)\n void reftable_ref_record_print(struct reftable_ref_record *ref,\n \t\t\t uint32_t hash_id)\n {\n-\tchar hex[2 * GIT_SHA256_RAWSZ + 1] = { 0 }; /* BUG */\n+\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */\n \tprintf(\"ref{%s(%\" PRIu64 \") \", ref->refname, ref->update_index);\n \tswitch (ref->value_type) {\n \tcase REFTABLE_REF_SYMREF:\n@@ -586,7 +586,7 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {\n void reftable_log_record_print(struct reftable_log_record *log,\n \t\t\t uint32_t hash_id)\n {\n-\tchar hex[GIT_SHA256_RAWSZ + 1] = { 0 };\n+\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 };\n \n \tswitch (log->value_type) {\n \tcase REFTABLE_LOG_DELETION:\n-- \ngitgitgadget\n\n"},{"id":"444796","messageId":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v4.git.git.1639482476.gitgitgadget@gmail.com","subject":"[PATCH v5 00/16] Reftable coverity fixes","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-22T18:56:20Z","receivedAt":"2021-12-22T18:56:42Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"This series was targeted to 'next'.\n\nThis addresses some complaints from Coverity that Peff reported.\n\nv5:\n\n * add generic record equality\n * add generic record printing\n * const correctness for record-reftable.h\n * fix windows crash.\n * 0-length memcpy paranoia\n * drop unused file.\n\nHan-Wen Nienhuys (16):\n reftable: fix OOB stack write in print functions\n reftable: fix resource leak in block.c error path\n reftable: fix resource leak blocksource.c\n reftable: check reftable_stack_auto_compact() return value\n reftable: ignore remove() return value in stack_test.c\n reftable: fix resource warning\n reftable: all xxx_free() functions accept NULL arguments\n reftable: order unittests by complexity\n reftable: drop stray printf in readwrite_test\n reftable: handle null refnames in reftable_ref_record_equal\n reftable: make reftable-record.h function signatures const correct\n reftable: implement record equality generically\n reftable: remove outdated file reftable.c\n reftable: make reftable_record a tagged union\n reftable: add print functions to the record types\n reftable: be more paranoid about 0-length memcpy calls\n\n reftable/block.c | 28 ++-\n reftable/block_test.c | 22 +--\n reftable/blocksource.c | 6 +-\n reftable/generic.c | 41 ++--\n reftable/iter.c | 4 +-\n reftable/merged.c | 33 ++--\n reftable/pq.c | 3 +-\n reftable/pq_test.c | 27 ++-\n reftable/reader.c | 113 ++++++-----\n reftable/readwrite_test.c | 67 ++++++-\n reftable/record.c | 376 ++++++++++++++++++++++++-------------\n reftable/record.h | 49 ++---\n reftable/record_test.c | 197 +++++++++----------\n reftable/reftable-record.h | 14 +-\n reftable/reftable.c | 115 ------------\n reftable/stack.c | 12 +-\n reftable/stack_test.c | 3 +-\n reftable/writer.c | 43 +++--\n t/helper/test-reftable.c | 9 +-\n 19 files changed, 629 insertions(+), 533 deletions(-)\n delete mode 100644 reftable/reftable.c\n\n\nbase-commit: 69a9c10c95e28df457e33b3c7400b16caf2e2962\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1152%2Fhanwen%2Freftable-coverity-v5\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1152/hanwen/reftable-coverity-v5\nPull-Request: https://github.com/git/git/pull/1152\n\nRange-diff vs v4:\n\n 1: 1c1a3ff92bd = 1: e7f1be7bbec reftable: fix OOB stack write in print functions\n 2: 923293b79b4 ! 2: 9ab631a3b29 reftable: fix resource leak in block.c error path\n @@ Metadata\n ## Commit message ##\n reftable: fix resource leak in block.c error path\n \n - Add test coverage for corrupt zlib data.\n + Add test coverage for corrupt zlib data. Fix memory leaks demonstrated by\n + unittest.\n \n This problem was discovered by a Coverity scan.\n \n @@ reftable/block.c: int block_reader_init(struct block_reader *br, struct reftable\n \n static uint32_t block_reader_restart_offset(struct block_reader *br, int i)\n \n + ## reftable/reader.c ##\n +@@ reftable/reader.c: int reader_init_block_reader(struct reftable_reader *r, struct block_reader *br,\n + \n + \terr = reader_get_block(r, &block, next_off, guess_block_size);\n + \tif (err < 0)\n +-\t\treturn err;\n ++\t\tgoto done;\n + \n + \tblock_size = extract_block_size(block.data, &block_typ, next_off,\n + \t\t\t\t\tr->version);\n +-\tif (block_size < 0)\n +-\t\treturn block_size;\n +-\n ++\tif (block_size < 0) {\n ++\t\terr = block_size;\n ++\t\tgoto done;\n ++\t}\n + \tif (want_typ != BLOCK_TYPE_ANY && block_typ != want_typ) {\n +-\t\treftable_block_done(&block);\n +-\t\treturn 1;\n ++\t\terr = 1;\n ++\t\tgoto done;\n + \t}\n + \n + \tif (block_size > guess_block_size) {\n + \t\treftable_block_done(&block);\n + \t\terr = reader_get_block(r, &block, next_off, block_size);\n + \t\tif (err < 0) {\n +-\t\t\treturn err;\n ++\t\t\tgoto done;\n + \t\t}\n + \t}\n + \n +-\treturn block_reader_init(br, &block, header_off, r->block_size,\n +-\t\t\t\t hash_size(r->hash_id));\n ++\terr = block_reader_init(br, &block, header_off, r->block_size,\n ++\t\t\t\thash_size(r->hash_id));\n ++done:\n ++\tif (err)\n ++\t\treftable_block_done(&block);\n ++\n ++\treturn err;\n + }\n + \n + static int table_iter_next_block(struct table_iter *dest,\n +\n ## reftable/readwrite_test.c ##\n @@ reftable/readwrite_test.c: static void test_log_write_read(void)\n \treader_close(&rd);\n 3: 7a914f77756 = 3: 8301000e83b reftable: fix resource leak blocksource.c\n 4: 51b4a17a2e1 = 4: e6e53aabc39 reftable: check reftable_stack_auto_compact() return value\n 5: 43989afcb5a = 5: 7551bcdd917 reftable: ignore remove() return value in stack_test.c\n 6: 1c7f15d811c = 6: 700a4e247e7 reftable: fix resource warning\n 7: 47ba5ddceb8 = 7: 30016fee4c8 reftable: all xxx_free() functions accept NULL arguments\n 8: aba8b8113ad = 8: f90846cc103 reftable: order unittests by complexity\n 9: f2af404d7f1 = 9: 425e12d3667 reftable: drop stray printf in readwrite_test\n 10: d2ce1bfef9f = 10: 67d858ec59e reftable: handle null refnames in reftable_ref_record_equal\n -: ----------- > 11: 1e50924894f reftable: make reftable-record.h function signatures const correct\n -: ----------- > 12: c81b17ad57e reftable: implement record equality generically\n -: ----------- > 13: 4175089ec43 reftable: remove outdated file reftable.c\n 11: 0ffc1cf0e42 ! 14: ede2e792ab6 reftable: make reftable_record a tagged union\n @@ reftable/generic.c: void reftable_iterator_destroy(struct reftable_iterator *it)\n -\tstruct reftable_record rec = { NULL };\n -\treftable_record_from_ref(&rec, ref);\n -\treturn iterator_next(it, &rec);\n -+\tstruct reftable_record rec = { .type = BLOCK_TYPE_REF };\n ++\tstruct reftable_record rec = {\n ++\t\t.type = BLOCK_TYPE_REF,\n ++\t\t.u.ref = *ref,\n ++\t};\n +\tint err = iterator_next(it, &rec);\n +\t*ref = rec.u.ref;\n +\treturn err;\n @@ reftable/generic.c: void reftable_iterator_destroy(struct reftable_iterator *it)\n -\tstruct reftable_record rec = { NULL };\n -\treftable_record_from_log(&rec, log);\n -\treturn iterator_next(it, &rec);\n -+\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG };\n ++\tstruct reftable_record rec = {\n ++\t\t.type = BLOCK_TYPE_LOG,\n ++\t\t.u.log = *log,\n ++\t};\n +\tint err = iterator_next(it, &rec);\n +\t*log = rec.u.log;\n +\treturn err;\n @@ reftable/merged.c: int reftable_merged_table_seek_log_at(struct reftable_merged_\n -\tstruct reftable_log_record log = {\n -\t\t.refname = (char *)name,\n -\t\t.update_index = update_index,\n -+\tstruct reftable_record rec = {\n -+\t\t.type = BLOCK_TYPE_LOG,\n -+\t\t.u.log = {\n -+\t\t\t.refname = (char *)name,\n -+\t\t\t.update_index = update_index,\n -+\t\t}\n - \t};\n +-\t};\n -\tstruct reftable_record rec = { NULL };\n -\treftable_record_from_log(&rec, &log);\n ++\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n ++\t\t\t\t .u.log = {\n ++\t\t\t\t\t .refname = (char *)name,\n ++\t\t\t\t\t .update_index = update_index,\n ++\t\t\t\t } };\n \treturn merged_table_seek_record(mt, it, &rec);\n }\n \n @@ reftable/pq_test.c: static void test_pq(void)\n -\n -\t\treftable_record_as_ref(&rec)->refname = names[i];\n -\t\te.rec = rec;\n -+\t\tstruct pq_entry e = {\n -+\t\t\t.rec = {\n -+\t\t\t\t.type = BLOCK_TYPE_REF,\n -+\t\t\t\t.u.ref = {\n -+\t\t\t\t\t.refname = names[i],\n -+\t\t\t\t}\n -+\t\t\t}\n -+\t\t};\n ++\t\tstruct pq_entry e = { .rec = { .type = BLOCK_TYPE_REF,\n ++\t\t\t\t\t .u.ref = {\n ++\t\t\t\t\t\t .refname = names[i],\n ++\t\t\t\t\t } } };\n \t\tmerged_iter_pqueue_add(&pq, e);\n \t\tmerged_iter_pqueue_check(pq);\n \t\ti = (i * 7) % N;\n @@ reftable/reader.c: static int table_iter_next_in_block(struct table_iter *ti,\n \t}\n \n \treturn res;\n -@@ reftable/reader.c: static int reader_start(struct reftable_reader *r, struct table_iter *ti,\n - static int reader_seek_linear(struct reftable_reader *r, struct table_iter *ti,\n - \t\t\t struct reftable_record *want)\n - {\n --\tstruct reftable_record rec =\n --\t\treftable_new_record(reftable_record_type(want));\n -+\tstruct reftable_record rec = reftable_new_record(reftable_record_type(want));\n - \tstruct strbuf want_key = STRBUF_INIT;\n - \tstruct strbuf got_key = STRBUF_INIT;\n - \tstruct table_iter next = TABLE_ITER_INIT;\n @@ reftable/reader.c: static int reader_seek_linear(struct reftable_reader *r, struct table_iter *ti,\n \n done:\n @@ reftable/reader.c: static int reader_seek_indexed(struct reftable_reader *r,\n -\tstruct reftable_index_record index_result = { .last_key = STRBUF_INIT };\n -\tstruct reftable_record index_result_rec = { NULL };\n +\tstruct reftable_record want_index = {\n -+\t\t.type = BLOCK_TYPE_INDEX,\n -+\t\t.u.idx = { .last_key = STRBUF_INIT }\n ++\t\t.type = BLOCK_TYPE_INDEX, .u.idx = { .last_key = STRBUF_INIT }\n +\t};\n +\tstruct reftable_record index_result = {\n +\t\t.type = BLOCK_TYPE_INDEX,\n @@ reftable/reader.c: static int reader_seek_indexed(struct reftable_reader *r,\n \treturn err;\n }\n \n -@@ reftable/reader.c: static int reader_seek_internal(struct reftable_reader *r,\n - \t\t\t\tstruct reftable_iterator *it,\n - \t\t\t\tstruct reftable_record *rec)\n - {\n --\tstruct reftable_reader_offsets *offs =\n --\t\treader_offsets_for(r, reftable_record_type(rec));\n -+\tstruct reftable_reader_offsets *offs = reader_offsets_for(r, reftable_record_type(rec));\n - \tuint64_t idx = offs->index_offset;\n - \tstruct table_iter ti = TABLE_ITER_INIT;\n - \tint err = 0;\n @@ reftable/reader.c: static int reader_seek(struct reftable_reader *r, struct reftable_iterator *it,\n int reftable_reader_seek_ref(struct reftable_reader *r,\n \t\t\t struct reftable_iterator *it, const char *name)\n @@ reftable/reader.c: int reftable_reader_seek_log_at(struct reftable_reader *r,\n -\tstruct reftable_log_record log = {\n -\t\t.refname = (char *)name,\n -\t\t.update_index = update_index,\n -+\tstruct reftable_record rec = {\n -+\t\t.type = BLOCK_TYPE_LOG,\n -+\t\t.u.log = {\n -+\t\t\t.refname = (char *)name,\n -+\t\t\t.update_index = update_index,\n -+\t\t}\n - \t};\n +-\t};\n -\tstruct reftable_record rec = { NULL };\n -\treftable_record_from_log(&rec, &log);\n ++\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n ++\t\t\t\t .u.log = {\n ++\t\t\t\t\t .refname = (char *)name,\n ++\t\t\t\t\t .update_index = update_index,\n ++\t\t\t\t } };\n \treturn reader_seek(r, it, &rec);\n }\n \n @@ reftable/reader.c: static int reftable_reader_refs_for_indexed(struct reftable_r\n \n \terr = new_indexed_table_ref_iter(&itr, r, oid, hash_size(r->hash_id),\n -\t\t\t\t\t got.offsets, got.offset_len);\n -+\t\t\t\t\t got.u.obj.offsets, got.u.obj.offset_len);\n ++\t\t\t\t\t got.u.obj.offsets,\n ++\t\t\t\t\t got.u.obj.offset_len);\n \tif (err < 0)\n \t\tgoto done;\n -\tgot.offsets = NULL;\n @@ reftable/record.c: https://developers.google.com/open-source/licenses/bsd\n int get_var_int(uint64_t *dest, struct string_view *in)\n {\n \tint ptr = 0;\n +@@ reftable/record.c: static void reftable_obj_record_copy_from(void *rec, const void *src_rec,\n + \t\t(const struct reftable_obj_record *)src_rec;\n + \n + \treftable_obj_record_release(obj);\n +-\t*obj = *src;\n +-\tobj->hash_prefix = reftable_malloc(obj->hash_prefix_len);\n +-\tmemcpy(obj->hash_prefix, src->hash_prefix, obj->hash_prefix_len);\n ++\tobj->hash_prefix = reftable_malloc(src->hash_prefix_len);\n ++\tobj->hash_prefix_len = src->hash_prefix_len;\n ++\tif (src->hash_prefix_len)\n ++\t\tmemcpy(obj->hash_prefix, src->hash_prefix, obj->hash_prefix_len);\n + \n +-\tobj->offsets = reftable_malloc(obj->offset_len * sizeof(uint64_t));\n +-\tCOPY_ARRAY(obj->offsets, src->offsets, obj->offset_len);\n ++\tobj->offsets = reftable_malloc(src->offset_len * sizeof(uint64_t));\n ++\tobj->offset_len = src->offset_len;\n ++\tCOPY_ARRAY(obj->offsets, src->offsets, src->offset_len);\n + }\n + \n + static uint8_t reftable_obj_record_val_type(const void *rec)\n @@ reftable/record.c: static struct reftable_record_vtable reftable_log_record_vtable = {\n - \t.is_deletion = &reftable_log_record_is_deletion_void,\n + \t.equal = &reftable_log_record_equal_void\n };\n \n -struct reftable_record reftable_new_record(uint8_t typ)\n @@ reftable/record.c: static struct reftable_record_vtable reftable_index_record_vt\n int reftable_record_is_deletion(struct reftable_record *rec)\n {\n -\treturn rec->ops->is_deletion(rec->data);\n ++\treturn reftable_record_vtable(rec)->is_deletion(\n ++\t\treftable_record_data(rec));\n + }\n + \n + int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size)\n + {\n +-\tif (a->ops != b->ops)\n ++\tif (a->type != b->type)\n + \t\treturn 0;\n +-\treturn a->ops->equal(a->data, b->data, hash_size);\n -}\n -\n -void reftable_record_from_ref(struct reftable_record *rec,\n @@ reftable/record.c: static struct reftable_record_vtable reftable_index_record_vt\n -{\n -\tassert(reftable_record_type(rec) == BLOCK_TYPE_LOG);\n -\treturn rec->data;\n -+\treturn reftable_record_vtable(rec)->is_deletion(\n -+\t\treftable_record_data(rec));\n ++\treturn reftable_record_vtable(a)->equal(\n ++\t\treftable_record_data(a), reftable_record_data(b), hash_size);\n }\n \n static int hash_equal(uint8_t *a, uint8_t *b, int hash_size)\n @@ reftable/record.c: void string_view_consume(struct string_view *s, int n)\n +\n +struct reftable_record reftable_new_record(uint8_t typ)\n +{\n -+\tstruct reftable_record clean_idx = {\n -+\t\t.type = BLOCK_TYPE_INDEX,\n -+\t\t.u.idx = {\n -+\t\t\t.last_key = STRBUF_INIT,\n -+\t\t},\n -+\t};\n +\tstruct reftable_record clean = {\n +\t\t.type = typ,\n +\t};\n +\n -+\tif (typ == BLOCK_TYPE_INDEX) {\n -+\t\treturn clean_idx;\n ++\t/* the following is involved, but the naive solution (just return\n ++\t * `clean` as is, except for BLOCK_TYPE_INDEX), returns a garbage\n ++\t * clean.u.obj.offsets pointer on Windows VS CI. Go figure.\n ++\t */\n ++\tswitch (typ) {\n ++\tcase BLOCK_TYPE_OBJ:\n ++\t{\n ++\t\tstruct reftable_obj_record obj = { 0 };\n ++\t\tclean.u.obj = obj;\n ++\t\tbreak;\n ++\t}\n ++\tcase BLOCK_TYPE_INDEX:\n ++\t{\n ++\t\tstruct reftable_index_record idx = {\n ++\t\t\t.last_key = STRBUF_INIT,\n ++\t\t};\n ++\t\tclean.u.idx = idx;\n ++\t\tbreak;\n ++\t}\n ++\tcase BLOCK_TYPE_REF:\n ++\t{\n ++\t\tstruct reftable_ref_record ref = { 0 };\n ++\t\tclean.u.ref = ref;\n ++\t\tbreak;\n ++\t}\n ++\tcase BLOCK_TYPE_LOG:\n ++\t{\n ++\t\tstruct reftable_log_record log = { 0 };\n ++\t\tclean.u.log = log;\n ++\t\tbreak;\n ++\t}\n +\t}\n -+\n +\treturn clean;\n +}\n \n ## reftable/record.h ##\n @@ reftable/record.h: struct reftable_record_vtable {\n - \tint (*is_deletion)(const void *rec);\n + \tint (*equal)(const void *a, const void *b, int hash_size);\n };\n \n -/* record is a generic wrapper for different types of records. */\n @@ reftable/record.h: struct reftable_obj_record {\n +};\n +\n /* see struct record_vtable */\n - \n + int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size);\n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest);\n @@ reftable/record.h: int reftable_record_decode(struct reftable_record *rec, struct strbuf key,\n \t\t\t int hash_size);\n @@ reftable/record_test.c\n {\n -\tstruct reftable_record copy =\n -\t\treftable_new_record(reftable_record_type(rec));\n -+\tstruct reftable_record copy = reftable_new_record(reftable_record_type(rec));\n ++\tstruct reftable_record copy = { 0 };\n ++\tuint8_t typ;\n +\n ++\ttyp = reftable_record_type(rec);\n ++\tcopy = reftable_new_record(typ);\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \t/* do it twice to catch memory leaks */\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n --\tswitch (reftable_record_type(©)) {\n -+\tswitch (copy.type) {\n - \tcase BLOCK_TYPE_REF:\n --\t\tEXPECT(reftable_ref_record_equal(reftable_record_as_ref(©),\n --\t\t\t\t\t\t reftable_record_as_ref(rec),\n -+\t\tEXPECT(reftable_ref_record_equal(©.u.ref, &rec->u.ref,\n - \t\t\t\t\t\t GIT_SHA1_RAWSZ));\n - \t\tbreak;\n - \tcase BLOCK_TYPE_LOG:\n --\t\tEXPECT(reftable_log_record_equal(reftable_record_as_log(©),\n --\t\t\t\t\t\t reftable_record_as_log(rec),\n -+\t\tEXPECT(reftable_log_record_equal(©.u.log, &rec->u.log,\n - \t\t\t\t\t\t GIT_SHA1_RAWSZ));\n - \t\tbreak;\n - \t}\n + \tEXPECT(reftable_record_equal(rec, ©, GIT_SHA1_RAWSZ));\n -\treftable_record_destroy(©);\n +\treftable_record_release(©);\n }\n @@ reftable/record_test.c: static void test_reftable_ref_record_roundtrip(void)\n +\t\tstruct reftable_record in = {\n +\t\t\t.type = BLOCK_TYPE_REF,\n +\t\t};\n -+\t\tstruct reftable_record out = {\n -+\t\t\t.type = BLOCK_TYPE_REF\n -+\t\t};\n ++\t\tstruct reftable_record out = { .type = BLOCK_TYPE_REF };\n \t\tstruct strbuf key = STRBUF_INIT;\n -\t\tstruct reftable_record rec = { NULL };\n \t\tuint8_t buffer[1024] = { 0 };\n @@ reftable/record_test.c: static void test_reftable_ref_record_roundtrip(void)\n -\t\t\tset_hash(in.value.val2.value, 1);\n -\t\t\tin.value.val2.target_value =\n +\t\t\tin.u.ref.value.val2.value =\n -+\t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n -+\t\t\tset_hash(in.u.ref.value.val2.value, 1);\n -+\t\t\tin.u.ref.value.val2.target_value =\n \t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n -\t\t\tset_hash(in.value.val2.target_value, 2);\n ++\t\t\tset_hash(in.u.ref.value.val2.value, 1);\n ++\t\t\tin.u.ref.value.val2.target_value =\n ++\t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n +\t\t\tset_hash(in.u.ref.value.val2.target_value, 2);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_SYMREF:\n @@ reftable/record_test.c: static void test_reftable_log_record_roundtrip(void)\n \tset_test_hash(in[0].value.update.old_hash, 2);\n \tfor (i = 0; i < ARRAY_SIZE(in); i++) {\n -\t\tstruct reftable_record rec = { NULL };\n -+\t\tstruct reftable_record rec = {\n -+\t\t\t.type = BLOCK_TYPE_LOG\n -+\t\t};\n ++\t\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG };\n \t\tstruct strbuf key = STRBUF_INIT;\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n @@ reftable/record_test.c: static void test_reftable_obj_record_roundtrip(void)\n \t\tstruct strbuf key = STRBUF_INIT;\n -\t\tstruct reftable_obj_record out = { NULL };\n -\t\tstruct reftable_record rec_out = { NULL };\n -+\t\tstruct reftable_record out = {\n -+\t\t\t.type = BLOCK_TYPE_OBJ\n -+\t\t};\n ++\t\tstruct reftable_record out = { .type = BLOCK_TYPE_OBJ };\n \t\tint n, m;\n \t\tuint8_t extra;\n \n @@ reftable/record_test.c: static void test_reftable_obj_record_roundtrip(void)\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n --\t\tEXPECT(in.hash_prefix_len == out.hash_prefix_len);\n --\t\tEXPECT(in.offset_len == out.offset_len);\n --\n --\t\tEXPECT(!memcmp(in.hash_prefix, out.hash_prefix,\n --\t\t\t in.hash_prefix_len));\n --\t\tEXPECT(0 == memcmp(in.offsets, out.offsets,\n --\t\t\t\t sizeof(uint64_t) * in.offset_len));\n -+\t\tEXPECT(in.u.obj.hash_prefix_len == out.u.obj.hash_prefix_len);\n -+\t\tEXPECT(in.u.obj.offset_len == out.u.obj.offset_len);\n -+\t\tif (in.u.obj.hash_prefix_len)\n -+\t\t\tEXPECT(!memcmp(in.u.obj.hash_prefix, out.u.obj.hash_prefix,\n -+\t\t\t\t in.u.obj.hash_prefix_len));\n -+\t\tif (in.u.obj.offset_len)\n -+\t\t\tEXPECT(!memcmp(in.u.obj.offsets, out.u.obj.offsets,\n -+\t\t\t\t\t sizeof(uint64_t) * in.u.obj.offset_len));\n +-\t\tEXPECT(reftable_record_equal(&rec, &rec_out, GIT_SHA1_RAWSZ));\n ++\t\tEXPECT(reftable_record_equal(&in, &out, GIT_SHA1_RAWSZ));\n \t\tstrbuf_release(&key);\n -\t\treftable_record_release(&rec_out);\n +\t\treftable_record_release(&out);\n @@ reftable/record_test.c: static void test_reftable_index_record_roundtrip(void)\n +\tm = reftable_record_decode(&out, key, extra, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(m == n);\n \n --\tEXPECT(in.offset == out.offset);\n -+\tEXPECT(in.u.idx.offset == out.u.idx.offset);\n +-\tEXPECT(reftable_record_equal(&rec, &out_rec, GIT_SHA1_RAWSZ));\n ++\tEXPECT(reftable_record_equal(&in, &out, GIT_SHA1_RAWSZ));\n \n -\treftable_record_release(&out_rec);\n +\treftable_record_release(&out);\n @@ reftable/writer.c: static void write_object_record(void *void_arg, void *key)\n -\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n -\t\t.offsets = entry->offsets,\n -\t\t.offset_len = entry->offset_len,\n -+\tstruct reftable_record rec = {\n -+\t\t.type = BLOCK_TYPE_OBJ,\n -+\t\t.u.obj = {\n -+\t\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n -+\t\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n -+\t\t\t.offsets = entry->offsets,\n -+\t\t\t.offset_len = entry->offset_len,\n -+\t\t}\n - \t};\n +-\t};\n -\tstruct reftable_record rec = { NULL };\n ++\tstruct reftable_record\n ++\t\trec = { .type = BLOCK_TYPE_OBJ,\n ++\t\t\t.u.obj = {\n ++\t\t\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n ++\t\t\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n ++\t\t\t\t.offsets = entry->offsets,\n ++\t\t\t\t.offset_len = entry->offset_len,\n ++\t\t\t} };\n \tif (arg->err < 0)\n \t\tgoto done;\n \n -: ----------- > 15: 6c06bd91662 reftable: add print functions to the record types\n -: ----------- > 16: e16bf0c5212 reftable: be more paranoid about 0-length memcpy calls\n\n-- \ngitgitgadget\n"},{"id":"444797","messageId":"9ab631a3b29addaa54415139e7f60a79a19a6edb.1640199396.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","subject":"[PATCH v5 02/16] reftable: fix resource leak in block.c error path","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-22T18:56:22Z","receivedAt":"2021-12-22T18:56:44Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nAdd test coverage for corrupt zlib data. Fix memory leaks demonstrated by\nunittest.\n\nThis problem was discovered by a Coverity scan.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/block.c | 26 +++++++++------\n reftable/reader.c | 24 ++++++++------\n reftable/readwrite_test.c | 66 +++++++++++++++++++++++++++++++++++++++\n 3 files changed, 98 insertions(+), 18 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 855e3f5c947..6c8e8705205 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -188,13 +188,16 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tuint32_t full_block_size = table_block_size;\n \tuint8_t typ = block->data[header_off];\n \tuint32_t sz = get_be24(block->data + header_off + 1);\n-\n+\tint err = 0;\n \tuint16_t restart_count = 0;\n \tuint32_t restart_start = 0;\n \tuint8_t *restart_bytes = NULL;\n+\tuint8_t *uncompressed = NULL;\n \n-\tif (!reftable_is_block_type(typ))\n-\t\treturn REFTABLE_FORMAT_ERROR;\n+\tif (!reftable_is_block_type(typ)) {\n+\t\terr = REFTABLE_FORMAT_ERROR;\n+\t\tgoto done;\n+\t}\n \n \tif (typ == BLOCK_TYPE_LOG) {\n \t\tint block_header_skip = 4 + header_off;\n@@ -203,7 +206,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\tuLongf src_len = block->len - block_header_skip;\n \t\t/* Log blocks specify the *uncompressed* size in their header.\n \t\t */\n-\t\tuint8_t *uncompressed = reftable_malloc(sz);\n+\t\tuncompressed = reftable_malloc(sz);\n \n \t\t/* Copy over the block header verbatim. It's not compressed. */\n \t\tmemcpy(uncompressed, block->data, block_header_skip);\n@@ -212,16 +215,19 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\tif (Z_OK !=\n \t\t uncompress2(uncompressed + block_header_skip, &dst_len,\n \t\t\t\tblock->data + block_header_skip, &src_len)) {\n-\t\t\treftable_free(uncompressed);\n-\t\t\treturn REFTABLE_ZLIB_ERROR;\n+\t\t\terr = REFTABLE_ZLIB_ERROR;\n+\t\t\tgoto done;\n \t\t}\n \n-\t\tif (dst_len + block_header_skip != sz)\n-\t\t\treturn REFTABLE_FORMAT_ERROR;\n+\t\tif (dst_len + block_header_skip != sz) {\n+\t\t\terr = REFTABLE_FORMAT_ERROR;\n+\t\t\tgoto done;\n+\t\t}\n \n \t\t/* We're done with the input data. */\n \t\treftable_block_done(block);\n \t\tblock->data = uncompressed;\n+\t\tuncompressed = NULL;\n \t\tblock->len = sz;\n \t\tblock->source = malloc_block_source();\n \t\tfull_block_size = src_len + block_header_skip;\n@@ -251,7 +257,9 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tbr->restart_count = restart_count;\n \tbr->restart_bytes = restart_bytes;\n \n-\treturn 0;\n+done:\n+\treftable_free(uncompressed);\n+\treturn err;\n }\n \n static uint32_t block_reader_restart_offset(struct block_reader *br, int i)\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 006709a645a..0d16b098f5e 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -290,28 +290,34 @@ int reader_init_block_reader(struct reftable_reader *r, struct block_reader *br,\n \n \terr = reader_get_block(r, &block, next_off, guess_block_size);\n \tif (err < 0)\n-\t\treturn err;\n+\t\tgoto done;\n \n \tblock_size = extract_block_size(block.data, &block_typ, next_off,\n \t\t\t\t\tr->version);\n-\tif (block_size < 0)\n-\t\treturn block_size;\n-\n+\tif (block_size < 0) {\n+\t\terr = block_size;\n+\t\tgoto done;\n+\t}\n \tif (want_typ != BLOCK_TYPE_ANY && block_typ != want_typ) {\n-\t\treftable_block_done(&block);\n-\t\treturn 1;\n+\t\terr = 1;\n+\t\tgoto done;\n \t}\n \n \tif (block_size > guess_block_size) {\n \t\treftable_block_done(&block);\n \t\terr = reader_get_block(r, &block, next_off, block_size);\n \t\tif (err < 0) {\n-\t\t\treturn err;\n+\t\t\tgoto done;\n \t\t}\n \t}\n \n-\treturn block_reader_init(br, &block, header_off, r->block_size,\n-\t\t\t\t hash_size(r->hash_id));\n+\terr = block_reader_init(br, &block, header_off, r->block_size,\n+\t\t\t\thash_size(r->hash_id));\n+done:\n+\tif (err)\n+\t\treftable_block_done(&block);\n+\n+\treturn err;\n }\n \n static int table_iter_next_block(struct table_iter *dest,\ndiff --git a/reftable/readwrite_test.c b/reftable/readwrite_test.c\nindex 5f6bcc2f775..6e88182a83a 100644\n--- a/reftable/readwrite_test.c\n+++ b/reftable/readwrite_test.c\n@@ -254,6 +254,71 @@ static void test_log_write_read(void)\n \treader_close(&rd);\n }\n \n+static void test_log_zlib_corruption(void)\n+{\n+\tstruct reftable_write_options opts = {\n+\t\t.block_size = 256,\n+\t};\n+\tstruct reftable_iterator it = { 0 };\n+\tstruct reftable_reader rd = { 0 };\n+\tstruct reftable_block_source source = { 0 };\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tstruct reftable_writer *w =\n+\t\treftable_new_writer(&strbuf_add_void, &buf, &opts);\n+\tconst struct reftable_stats *stats = NULL;\n+\tuint8_t hash1[GIT_SHA1_RAWSZ] = { 1 };\n+\tuint8_t hash2[GIT_SHA1_RAWSZ] = { 2 };\n+\tchar message[100] = { 0 };\n+\tint err, i, n;\n+\n+\tstruct reftable_log_record log = {\n+\t\t.refname = \"refname\",\n+\t\t.value_type = REFTABLE_LOG_UPDATE,\n+\t\t.value = {\n+\t\t\t.update = {\n+\t\t\t\t.new_hash = hash1,\n+\t\t\t\t.old_hash = hash2,\n+\t\t\t\t.name = \"My Name\",\n+\t\t\t\t.email = \"myname@invalid\",\n+\t\t\t\t.message = message,\n+\t\t\t},\n+\t\t},\n+\t};\n+\n+\tfor (i = 0; i < sizeof(message)-1; i++)\n+\t\tmessage[i] = (uint8_t)(rand() % 64 + ' ');\n+\n+\treftable_writer_set_limits(w, 1, 1);\n+\n+\terr = reftable_writer_add_log(w, &log);\n+\tEXPECT_ERR(err);\n+\n+\tn = reftable_writer_close(w);\n+\tEXPECT(n == 0);\n+\n+\tstats = writer_stats(w);\n+\tEXPECT(stats->log_stats.blocks > 0);\n+\treftable_writer_free(w);\n+\tw = NULL;\n+\n+\t/* corrupt the data. */\n+\tbuf.buf[50] ^= 0x99;\n+\n+\tblock_source_from_strbuf(&source, &buf);\n+\n+\terr = init_reader(&rd, &source, \"file.log\");\n+\tEXPECT_ERR(err);\n+\n+\terr = reftable_reader_seek_log(&rd, &it, \"refname\");\n+\tEXPECT(err == REFTABLE_ZLIB_ERROR);\n+\n+\treftable_iterator_destroy(&it);\n+\n+\t/* cleanup. */\n+\tstrbuf_release(&buf);\n+\treader_close(&rd);\n+}\n+\n static void test_table_read_write_sequential(void)\n {\n \tchar **names;\n@@ -633,6 +698,7 @@ static void test_corrupt_table(void)\n \n int readwrite_test_main(int argc, const char *argv[])\n {\n+\tRUN_TEST(test_log_zlib_corruption);\n \tRUN_TEST(test_corrupt_table);\n \tRUN_TEST(test_corrupt_table_empty);\n \tRUN_TEST(test_log_write_read);\n-- \ngitgitgadget\n\n"},{"id":"444798","messageId":"8301000e83b2f9758087bb741aeac0e6238ad343.1640199396.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","subject":"[PATCH v5 03/16] reftable: fix resource leak blocksource.c","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-22T18:56:23Z","receivedAt":"2021-12-22T18:56:45Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis would be triggered in the unlikely event of fstat() failing on an opened\nfile.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/blocksource.c | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/reftable/blocksource.c b/reftable/blocksource.c\nindex 0044eecd9aa..2605371c28d 100644\n--- a/reftable/blocksource.c\n+++ b/reftable/blocksource.c\n@@ -134,8 +134,10 @@ int reftable_block_source_from_file(struct reftable_block_source *bs,\n \t}\n \n \terr = fstat(fd, &st);\n-\tif (err < 0)\n-\t\treturn -1;\n+\tif (err < 0) {\n+\t\tclose(fd);\n+\t\treturn REFTABLE_IO_ERROR;\n+\t}\n \n \tp = reftable_calloc(sizeof(struct file_block_source));\n \tp->size = st.st_size;\n-- \ngitgitgadget\n\n"},{"id":"444799","messageId":"e6e53aabc39a347b7dfd8bd601e2e2a6e4b37268.1640199396.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","subject":"[PATCH v5 04/16] reftable: check reftable_stack_auto_compact() return value","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-22T18:56:24Z","receivedAt":"2021-12-22T18:56:46Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nFixes a problem detected by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack_test.c | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/reftable/stack_test.c b/reftable/stack_test.c\nindex eb0b7228b0c..d628420e63a 100644\n--- a/reftable/stack_test.c\n+++ b/reftable/stack_test.c\n@@ -814,6 +814,7 @@ static void test_reftable_stack_auto_compaction(void)\n \t\tEXPECT_ERR(err);\n \n \t\terr = reftable_stack_auto_compact(st);\n+\t\tEXPECT_ERR(err);\n \t\tEXPECT(i < 3 || st->merged->stack_len < 2 * fastlog2(i));\n \t}\n \n-- \ngitgitgadget\n\n"},{"id":"444800","messageId":"7551bcdd9175d0e7a0bdc60ad0fed7310976e02d.1640199396.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","subject":"[PATCH v5 05/16] reftable: ignore remove() return value in stack_test.c","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-22T18:56:25Z","receivedAt":"2021-12-22T18:56:47Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nIf the cleanup fails, there is nothing we can do.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack_test.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/reftable/stack_test.c b/reftable/stack_test.c\nindex d628420e63a..4b7292945c3 100644\n--- a/reftable/stack_test.c\n+++ b/reftable/stack_test.c\n@@ -89,7 +89,7 @@ static void test_read_file(void)\n \t\tEXPECT(0 == strcmp(want[i], names[i]));\n \t}\n \tfree_names(names);\n-\tremove(fn);\n+\t(void) remove(fn);\n }\n \n static void test_parse_names(void)\n-- \ngitgitgadget\n\n"},{"id":"444801","messageId":"700a4e247e7e1deb19358960bbb75ebbde88ddbd.1640199396.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","subject":"[PATCH v5 06/16] reftable: fix resource warning","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-22T18:56:26Z","receivedAt":"2021-12-22T18:56:49Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis would trigger in the unlikely event that we are compacting, and the next\navailable file handle is 0.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack.c | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/reftable/stack.c b/reftable/stack.c\nindex df5021ebf08..10dfd370e8e 100644\n--- a/reftable/stack.c\n+++ b/reftable/stack.c\n@@ -877,7 +877,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \tstruct strbuf new_table_path = STRBUF_INIT;\n \tint err = 0;\n \tint have_lock = 0;\n-\tint lock_file_fd = 0;\n+\tint lock_file_fd = -1;\n \tint compact_count = last - first + 1;\n \tchar **listp = NULL;\n \tchar **delete_on_success =\n@@ -911,7 +911,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \t}\n \t/* Don't want to write to the lock for now. */\n \tclose(lock_file_fd);\n-\tlock_file_fd = 0;\n+\tlock_file_fd = -1;\n \n \thave_lock = 1;\n \terr = stack_uptodate(st);\n@@ -932,7 +932,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \n \t\tsublock_file_fd = open(subtab_lock.buf,\n \t\t\t\t O_EXCL | O_CREAT | O_WRONLY, 0644);\n-\t\tif (sublock_file_fd > 0) {\n+\t\tif (sublock_file_fd >= 0) {\n \t\t\tclose(sublock_file_fd);\n \t\t} else if (sublock_file_fd < 0) {\n \t\t\tif (errno == EEXIST) {\n@@ -1013,7 +1013,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \t\tgoto done;\n \t}\n \terr = close(lock_file_fd);\n-\tlock_file_fd = 0;\n+\tlock_file_fd = -1;\n \tif (err < 0) {\n \t\terr = REFTABLE_IO_ERROR;\n \t\tunlink(new_table_path.buf);\n@@ -1050,9 +1050,9 @@ done:\n \t\tlistp++;\n \t}\n \tfree_names(subtable_locks);\n-\tif (lock_file_fd > 0) {\n+\tif (lock_file_fd >= 0) {\n \t\tclose(lock_file_fd);\n-\t\tlock_file_fd = 0;\n+\t\tlock_file_fd = -1;\n \t}\n \tif (have_lock) {\n \t\tunlink(lock_file_name.buf);\n-- \ngitgitgadget\n\n"},{"id":"444802","messageId":"f90846cc103b8b04ff62ce1f942ee1cd0383573c.1640199396.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","subject":"[PATCH v5 08/16] reftable: order unittests by complexity","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-22T18:56:28Z","receivedAt":"2021-12-22T18:56:54Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis is a more practical ordering when working on refactorings of the reftable\ncode.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n t/helper/test-reftable.c | 9 +++++----\n 1 file changed, 5 insertions(+), 4 deletions(-)\n\ndiff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c\nindex 26b03d7b789..1f0a28cbb64 100644\n--- a/t/helper/test-reftable.c\n+++ b/t/helper/test-reftable.c\n@@ -3,15 +3,16 @@\n \n int cmd__reftable(int argc, const char **argv)\n {\n+\t/* test from simple to complex. */\n \tbasics_test_main(argc, argv);\n+\trecord_test_main(argc, argv);\n \tblock_test_main(argc, argv);\n-\tmerged_test_main(argc, argv);\n+\ttree_test_main(argc, argv);\n \tpq_test_main(argc, argv);\n-\trecord_test_main(argc, argv);\n-\trefname_test_main(argc, argv);\n \treadwrite_test_main(argc, argv);\n+\tmerged_test_main(argc, argv);\n \tstack_test_main(argc, argv);\n-\ttree_test_main(argc, argv);\n+\trefname_test_main(argc, argv);\n \treturn 0;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"444803","messageId":"30016fee4c85c0a803017070d28b3ccf2111ee91.1640199396.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","subject":"[PATCH v5 07/16] reftable: all xxx_free() functions accept NULL arguments","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-22T18:56:27Z","receivedAt":"2021-12-22T18:56:55Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis fixes NULL derefs in error paths. Spotted by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/reader.c | 2 ++\n reftable/writer.c | 2 ++\n 2 files changed, 4 insertions(+)\n\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 0d16b098f5e..c7a32331c76 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -647,6 +647,8 @@ int reftable_new_reader(struct reftable_reader **p,\n \n void reftable_reader_free(struct reftable_reader *r)\n {\n+\tif (!r)\n+\t\treturn;\n \treader_close(r);\n \treftable_free(r);\n }\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex 3ca721e9f64..9fd24fa93ce 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -150,6 +150,8 @@ void reftable_writer_set_limits(struct reftable_writer *w, uint64_t min,\n \n void reftable_writer_free(struct reftable_writer *w)\n {\n+\tif (!w)\n+\t\treturn;\n \treftable_free(w->block);\n \treftable_free(w);\n }\n-- \ngitgitgadget\n\n"},{"id":"444804","messageId":"425e12d3667754a152b32a75cad435aad6a06148.1640199396.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","subject":"[PATCH v5 09/16] reftable: drop stray printf in readwrite_test","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-22T18:56:29Z","receivedAt":"2021-12-22T18:56:56Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/readwrite_test.c | 1 -\n 1 file changed, 1 deletion(-)\n\ndiff --git a/reftable/readwrite_test.c b/reftable/readwrite_test.c\nindex 6e88182a83a..9b89a5da103 100644\n--- a/reftable/readwrite_test.c\n+++ b/reftable/readwrite_test.c\n@@ -662,7 +662,6 @@ static void test_write_key_order(void)\n \terr = reftable_writer_add_ref(w, &refs[0]);\n \tEXPECT_ERR(err);\n \terr = reftable_writer_add_ref(w, &refs[1]);\n-\tprintf(\"%d\\n\", err);\n \tEXPECT(err == REFTABLE_API_ERROR);\n \treftable_writer_close(w);\n \treftable_writer_free(w);\n-- \ngitgitgadget\n\n"},{"id":"444805","messageId":"67d858ec59e1c62d5e227e85e117b3cfcb3dda2a.1640199396.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","subject":"[PATCH v5 10/16] reftable: handle null refnames in reftable_ref_record_equal","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-22T18:56:30Z","receivedAt":"2021-12-22T18:56:57Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSpotted by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 8536bd03aa9..8bbcbff1e69 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -1154,9 +1154,11 @@ int reftable_ref_record_equal(struct reftable_ref_record *a,\n \t\t\t struct reftable_ref_record *b, int hash_size)\n {\n \tassert(hash_size > 0);\n-\tif (!(0 == strcmp(a->refname, b->refname) &&\n-\t a->update_index == b->update_index &&\n-\t a->value_type == b->value_type))\n+\tif (!null_streq(a->refname, b->refname))\n+\t\treturn 0;\n+\n+\tif (a->update_index != b->update_index ||\n+\t a->value_type != b->value_type)\n \t\treturn 0;\n \n \tswitch (a->value_type) {\n-- \ngitgitgadget\n\n"},{"id":"444806","messageId":"1e50924894fcc07ee74a88a8b9f44547db01a3df.1640199396.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","subject":"[PATCH v5 11/16] reftable: make reftable-record.h function signatures const correct","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-22T18:56:31Z","receivedAt":"2021-12-22T18:56:58Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 14 +++++++-------\n reftable/reftable-record.h | 14 +++++++-------\n 2 files changed, 14 insertions(+), 14 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 8bbcbff1e69..f7c77c51539 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -72,7 +72,7 @@ int reftable_is_block_type(uint8_t typ)\n \treturn 0;\n }\n \n-uint8_t *reftable_ref_record_val1(struct reftable_ref_record *rec)\n+uint8_t *reftable_ref_record_val1(const struct reftable_ref_record *rec)\n {\n \tswitch (rec->value_type) {\n \tcase REFTABLE_REF_VAL1:\n@@ -84,7 +84,7 @@ uint8_t *reftable_ref_record_val1(struct reftable_ref_record *rec)\n \t}\n }\n \n-uint8_t *reftable_ref_record_val2(struct reftable_ref_record *rec)\n+uint8_t *reftable_ref_record_val2(const struct reftable_ref_record *rec)\n {\n \tswitch (rec->value_type) {\n \tcase REFTABLE_REF_VAL2:\n@@ -251,7 +251,7 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)\n \t}\n }\n \n-void reftable_ref_record_print(struct reftable_ref_record *ref,\n+void reftable_ref_record_print(const struct reftable_ref_record *ref,\n \t\t\t uint32_t hash_id)\n {\n \tchar hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */\n@@ -881,8 +881,8 @@ static int zero_hash_eq(uint8_t *a, uint8_t *b, int sz)\n \treturn !memcmp(a, b, sz);\n }\n \n-int reftable_log_record_equal(struct reftable_log_record *a,\n-\t\t\t struct reftable_log_record *b, int hash_size)\n+int reftable_log_record_equal(const struct reftable_log_record *a,\n+\t\t\t const struct reftable_log_record *b, int hash_size)\n {\n \tif (!(null_streq(a->refname, b->refname) &&\n \t a->update_index == b->update_index &&\n@@ -1150,8 +1150,8 @@ static int hash_equal(uint8_t *a, uint8_t *b, int hash_size)\n \treturn a == b;\n }\n \n-int reftable_ref_record_equal(struct reftable_ref_record *a,\n-\t\t\t struct reftable_ref_record *b, int hash_size)\n+int reftable_ref_record_equal(const struct reftable_ref_record *a,\n+\t\t\t const struct reftable_ref_record *b, int hash_size)\n {\n \tassert(hash_size > 0);\n \tif (!null_streq(a->refname, b->refname))\ndiff --git a/reftable/reftable-record.h b/reftable/reftable-record.h\nindex 5370d2288c7..67104f8fbfe 100644\n--- a/reftable/reftable-record.h\n+++ b/reftable/reftable-record.h\n@@ -49,25 +49,25 @@ struct reftable_ref_record {\n \n /* Returns the first hash, or NULL if `rec` is not of type\n * REFTABLE_REF_VAL1 or REFTABLE_REF_VAL2. */\n-uint8_t *reftable_ref_record_val1(struct reftable_ref_record *rec);\n+uint8_t *reftable_ref_record_val1(const struct reftable_ref_record *rec);\n \n /* Returns the second hash, or NULL if `rec` is not of type\n * REFTABLE_REF_VAL2. */\n-uint8_t *reftable_ref_record_val2(struct reftable_ref_record *rec);\n+uint8_t *reftable_ref_record_val2(const struct reftable_ref_record *rec);\n \n /* returns whether 'ref' represents a deletion */\n int reftable_ref_record_is_deletion(const struct reftable_ref_record *ref);\n \n /* prints a reftable_ref_record onto stdout. Useful for debugging. */\n-void reftable_ref_record_print(struct reftable_ref_record *ref,\n+void reftable_ref_record_print(const struct reftable_ref_record *ref,\n \t\t\t uint32_t hash_id);\n \n /* frees and nulls all pointer values inside `ref`. */\n void reftable_ref_record_release(struct reftable_ref_record *ref);\n \n /* returns whether two reftable_ref_records are the same. Useful for testing. */\n-int reftable_ref_record_equal(struct reftable_ref_record *a,\n-\t\t\t struct reftable_ref_record *b, int hash_size);\n+int reftable_ref_record_equal(const struct reftable_ref_record *a,\n+\t\t\t const struct reftable_ref_record *b, int hash_size);\n \n /* reftable_log_record holds a reflog entry */\n struct reftable_log_record {\n@@ -104,8 +104,8 @@ int reftable_log_record_is_deletion(const struct reftable_log_record *log);\n void reftable_log_record_release(struct reftable_log_record *log);\n \n /* returns whether two records are equal. Useful for testing. */\n-int reftable_log_record_equal(struct reftable_log_record *a,\n-\t\t\t struct reftable_log_record *b, int hash_size);\n+int reftable_log_record_equal(const struct reftable_log_record *a,\n+\t\t\t const struct reftable_log_record *b, int hash_size);\n \n /* dumps a reftable_log_record on stdout, for debugging/testing. */\n void reftable_log_record_print(struct reftable_log_record *log,\n-- \ngitgitgadget\n\n"},{"id":"444807","messageId":"c81b17ad57e2ac82bca8d8705099fc6d343cae81.1640199396.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","subject":"[PATCH v5 12/16] reftable: implement record equality generically","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-22T18:56:32Z","receivedAt":"2021-12-22T18:56:59Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis simplifies unittests a little, and provides further coverage for\nreftable_record_copy().\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 57 +++++++++++++++++++++++++++++++++++++++++-\n reftable/record.h | 5 +++-\n reftable/record_test.c | 23 +++--------------\n 3 files changed, 63 insertions(+), 22 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex f7c77c51539..2a9e41a992e 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -430,6 +430,15 @@ static int reftable_ref_record_is_deletion_void(const void *p)\n \t\t(const struct reftable_ref_record *)p);\n }\n \n+\n+static int reftable_ref_record_equal_void(const void *a,\n+\t\t\t\t\t const void *b, int hash_size)\n+{\n+\tstruct reftable_ref_record *ra = (struct reftable_ref_record *) a;\n+\tstruct reftable_ref_record *rb = (struct reftable_ref_record *) b;\n+\treturn reftable_ref_record_equal(ra, rb, hash_size);\n+}\n+\n static struct reftable_record_vtable reftable_ref_record_vtable = {\n \t.key = &reftable_ref_record_key,\n \t.type = BLOCK_TYPE_REF,\n@@ -439,6 +448,7 @@ static struct reftable_record_vtable reftable_ref_record_vtable = {\n \t.decode = &reftable_ref_record_decode,\n \t.release = &reftable_ref_record_release_void,\n \t.is_deletion = &reftable_ref_record_is_deletion_void,\n+\t.equal = &reftable_ref_record_equal_void,\n };\n \n static void reftable_obj_record_key(const void *r, struct strbuf *dest)\n@@ -572,6 +582,25 @@ static int not_a_deletion(const void *p)\n \treturn 0;\n }\n \n+static int reftable_obj_record_equal_void(const void *a, const void *b, int hash_size)\n+{\n+\tstruct reftable_obj_record *ra = (struct reftable_obj_record *) a;\n+\tstruct reftable_obj_record *rb = (struct reftable_obj_record *) b;\n+\n+\tif (ra->hash_prefix_len != rb->hash_prefix_len\n+\t || ra->offset_len != rb->offset_len)\n+\t\treturn 0;\n+\n+\tif (ra->hash_prefix_len &&\n+\t memcmp(ra->hash_prefix, rb->hash_prefix, ra->hash_prefix_len))\n+\t\treturn 0;\n+\tif (ra->offset_len &&\n+\t memcmp(ra->offsets, rb->offsets, ra->offset_len * sizeof(uint64_t)))\n+\t\treturn 0;\n+\n+\treturn 1;\n+}\n+\n static struct reftable_record_vtable reftable_obj_record_vtable = {\n \t.key = &reftable_obj_record_key,\n \t.type = BLOCK_TYPE_OBJ,\n@@ -580,7 +609,8 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {\n \t.encode = &reftable_obj_record_encode,\n \t.decode = &reftable_obj_record_decode,\n \t.release = &reftable_obj_record_release,\n-\t.is_deletion = not_a_deletion,\n+\t.is_deletion = ¬_a_deletion,\n+\t.equal = &reftable_obj_record_equal_void,\n };\n \n void reftable_log_record_print(struct reftable_log_record *log,\n@@ -881,6 +911,14 @@ static int zero_hash_eq(uint8_t *a, uint8_t *b, int sz)\n \treturn !memcmp(a, b, sz);\n }\n \n+static int reftable_log_record_equal_void(const void *a,\n+\t\t\t\t\t const void *b, int hash_size)\n+{\n+\treturn reftable_log_record_equal((struct reftable_log_record *) a,\n+\t\t\t\t\t (struct reftable_log_record *) b,\n+\t\t\t\t\t hash_size);\n+}\n+\n int reftable_log_record_equal(const struct reftable_log_record *a,\n \t\t\t const struct reftable_log_record *b, int hash_size)\n {\n@@ -924,6 +962,7 @@ static struct reftable_record_vtable reftable_log_record_vtable = {\n \t.decode = &reftable_log_record_decode,\n \t.release = &reftable_log_record_release_void,\n \t.is_deletion = &reftable_log_record_is_deletion_void,\n+\t.equal = &reftable_log_record_equal_void\n };\n \n struct reftable_record reftable_new_record(uint8_t typ)\n@@ -1042,6 +1081,14 @@ static int reftable_index_record_decode(void *rec, struct strbuf key,\n \treturn start.len - in.len;\n }\n \n+static int reftable_index_record_equal(const void *a, const void *b, int hash_size)\n+{\n+\tstruct reftable_index_record *ia = (struct reftable_index_record *) a;\n+\tstruct reftable_index_record *ib = (struct reftable_index_record *) b;\n+\n+\treturn ia->offset == ib->offset && !strbuf_cmp(&ia->last_key, &ib->last_key);\n+}\n+\n static struct reftable_record_vtable reftable_index_record_vtable = {\n \t.key = &reftable_index_record_key,\n \t.type = BLOCK_TYPE_INDEX,\n@@ -1051,6 +1098,7 @@ static struct reftable_record_vtable reftable_index_record_vtable = {\n \t.decode = &reftable_index_record_decode,\n \t.release = &reftable_index_record_release,\n \t.is_deletion = ¬_a_deletion,\n+\t.equal = &reftable_index_record_equal,\n };\n \n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)\n@@ -1098,6 +1146,13 @@ int reftable_record_is_deletion(struct reftable_record *rec)\n \treturn rec->ops->is_deletion(rec->data);\n }\n \n+int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size)\n+{\n+\tif (a->ops != b->ops)\n+\t\treturn 0;\n+\treturn a->ops->equal(a->data, b->data, hash_size);\n+}\n+\n void reftable_record_from_ref(struct reftable_record *rec,\n \t\t\t struct reftable_ref_record *ref_rec)\n {\ndiff --git a/reftable/record.h b/reftable/record.h\nindex 498e8c50bf4..da75d7d1f11 100644\n--- a/reftable/record.h\n+++ b/reftable/record.h\n@@ -58,6 +58,9 @@ struct reftable_record_vtable {\n \n \t/* is this a tombstone? */\n \tint (*is_deletion)(const void *rec);\n+\n+\t/* Are two records equal? This assumes they have the same type. Returns 0 for non-equal. */\n+\tint (*equal)(const void *a, const void *b, int hash_size);\n };\n \n /* record is a generic wrapper for different types of records. */\n@@ -98,7 +101,7 @@ struct reftable_obj_record {\n };\n \n /* see struct record_vtable */\n-\n+int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size);\n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest);\n uint8_t reftable_record_type(struct reftable_record *rec);\n void reftable_record_copy_from(struct reftable_record *rec,\ndiff --git a/reftable/record_test.c b/reftable/record_test.c\nindex f4ad7cace41..92680848156 100644\n--- a/reftable/record_test.c\n+++ b/reftable/record_test.c\n@@ -21,18 +21,7 @@ static void test_copy(struct reftable_record *rec)\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \t/* do it twice to catch memory leaks */\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n-\tswitch (reftable_record_type(©)) {\n-\tcase BLOCK_TYPE_REF:\n-\t\tEXPECT(reftable_ref_record_equal(reftable_record_as_ref(©),\n-\t\t\t\t\t\t reftable_record_as_ref(rec),\n-\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n-\t\tbreak;\n-\tcase BLOCK_TYPE_LOG:\n-\t\tEXPECT(reftable_log_record_equal(reftable_record_as_log(©),\n-\t\t\t\t\t\t reftable_record_as_log(rec),\n-\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n-\t\tbreak;\n-\t}\n+\tEXPECT(reftable_record_equal(rec, ©, GIT_SHA1_RAWSZ));\n \treftable_record_destroy(©);\n }\n \n@@ -346,13 +335,7 @@ static void test_reftable_obj_record_roundtrip(void)\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(in.hash_prefix_len == out.hash_prefix_len);\n-\t\tEXPECT(in.offset_len == out.offset_len);\n-\n-\t\tEXPECT(!memcmp(in.hash_prefix, out.hash_prefix,\n-\t\t\t in.hash_prefix_len));\n-\t\tEXPECT(0 == memcmp(in.offsets, out.offsets,\n-\t\t\t\t sizeof(uint64_t) * in.offset_len));\n+\t\tEXPECT(reftable_record_equal(&rec, &rec_out, GIT_SHA1_RAWSZ));\n \t\tstrbuf_release(&key);\n \t\treftable_record_release(&rec_out);\n \t}\n@@ -390,7 +373,7 @@ static void test_reftable_index_record_roundtrip(void)\n \tm = reftable_record_decode(&out_rec, key, extra, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(m == n);\n \n-\tEXPECT(in.offset == out.offset);\n+\tEXPECT(reftable_record_equal(&rec, &out_rec, GIT_SHA1_RAWSZ));\n \n \treftable_record_release(&out_rec);\n \tstrbuf_release(&key);\n-- \ngitgitgadget\n\n"},{"id":"444808","messageId":"4175089ec432da921d158fec9ccb1902be710af6.1640199396.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","subject":"[PATCH v5 13/16] reftable: remove outdated file reftable.c","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-22T18:56:33Z","receivedAt":"2021-12-22T18:57:05Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis was renamed to generic.c, but the origin was never removed\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/reftable.c | 115 --------------------------------------------\n 1 file changed, 115 deletions(-)\n delete mode 100644 reftable/reftable.c\n\ndiff --git a/reftable/reftable.c b/reftable/reftable.c\ndeleted file mode 100644\nindex 0e4607a7cd6..00000000000\n--- a/reftable/reftable.c\n+++ /dev/null\n@@ -1,115 +0,0 @@\n-/*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n-\n-#include \"basics.h\"\n-#include \"record.h\"\n-#include \"generic.h\"\n-#include \"reftable-iterator.h\"\n-#include \"reftable-generic.h\"\n-\n-int reftable_table_seek_ref(struct reftable_table *tab,\n-\t\t\t struct reftable_iterator *it, const char *name)\n-{\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n-\treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n-}\n-\n-int reftable_table_read_ref(struct reftable_table *tab, const char *name,\n-\t\t\t struct reftable_ref_record *ref)\n-{\n-\tstruct reftable_iterator it = { NULL };\n-\tint err = reftable_table_seek_ref(tab, &it, name);\n-\tif (err)\n-\t\tgoto done;\n-\n-\terr = reftable_iterator_next_ref(&it, ref);\n-\tif (err)\n-\t\tgoto done;\n-\n-\tif (strcmp(ref->refname, name) ||\n-\t reftable_ref_record_is_deletion(ref)) {\n-\t\treftable_ref_record_release(ref);\n-\t\terr = 1;\n-\t\tgoto done;\n-\t}\n-\n-done:\n-\treftable_iterator_destroy(&it);\n-\treturn err;\n-}\n-\n-uint64_t reftable_table_max_update_index(struct reftable_table *tab)\n-{\n-\treturn tab->ops->max_update_index(tab->table_arg);\n-}\n-\n-uint64_t reftable_table_min_update_index(struct reftable_table *tab)\n-{\n-\treturn tab->ops->min_update_index(tab->table_arg);\n-}\n-\n-uint32_t reftable_table_hash_id(struct reftable_table *tab)\n-{\n-\treturn tab->ops->hash_id(tab->table_arg);\n-}\n-\n-void reftable_iterator_destroy(struct reftable_iterator *it)\n-{\n-\tif (!it->ops) {\n-\t\treturn;\n-\t}\n-\tit->ops->close(it->iter_arg);\n-\tit->ops = NULL;\n-\tFREE_AND_NULL(it->iter_arg);\n-}\n-\n-int reftable_iterator_next_ref(struct reftable_iterator *it,\n-\t\t\t struct reftable_ref_record *ref)\n-{\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, ref);\n-\treturn iterator_next(it, &rec);\n-}\n-\n-int reftable_iterator_next_log(struct reftable_iterator *it,\n-\t\t\t struct reftable_log_record *log)\n-{\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, log);\n-\treturn iterator_next(it, &rec);\n-}\n-\n-int iterator_next(struct reftable_iterator *it, struct reftable_record *rec)\n-{\n-\treturn it->ops->next(it->iter_arg, rec);\n-}\n-\n-static int empty_iterator_next(void *arg, struct reftable_record *rec)\n-{\n-\treturn 1;\n-}\n-\n-static void empty_iterator_close(void *arg)\n-{\n-}\n-\n-static struct reftable_iterator_vtable empty_vtable = {\n-\t.next = &empty_iterator_next,\n-\t.close = &empty_iterator_close,\n-};\n-\n-void iterator_set_empty(struct reftable_iterator *it)\n-{\n-\tassert(!it->ops);\n-\tit->iter_arg = NULL;\n-\tit->ops = &empty_vtable;\n-}\n-- \ngitgitgadget\n\n"},{"id":"444809","messageId":"6c06bd91662011fa115d1e5b2420ac0d315befb6.1640199396.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","subject":"[PATCH v5 15/16] reftable: add print functions to the record types","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-22T18:56:35Z","receivedAt":"2021-12-22T18:57:06Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis isn't used per se, but it is useful for debugging, especially\nWindows CI failures.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 85 +++++++++++++++++++++++++++++++++++-------\n reftable/record.h | 4 ++\n reftable/record_test.c | 21 ++++++++++-\n 3 files changed, 95 insertions(+), 15 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex a8cee628942..fbaa1fbef56 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -255,8 +255,8 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)\n \t}\n }\n \n-void reftable_ref_record_print(const struct reftable_ref_record *ref,\n-\t\t\t uint32_t hash_id)\n+static void reftable_ref_record_print_sz(const struct reftable_ref_record *ref,\n+\t\t\t\t\t int hash_size)\n {\n \tchar hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */\n \tprintf(\"ref{%s(%\" PRIu64 \") \", ref->refname, ref->update_index);\n@@ -265,14 +265,14 @@ void reftable_ref_record_print(const struct reftable_ref_record *ref,\n \t\tprintf(\"=> %s\", ref->value.symref);\n \t\tbreak;\n \tcase REFTABLE_REF_VAL2:\n-\t\thex_format(hex, ref->value.val2.value, hash_size(hash_id));\n+\t\thex_format(hex, ref->value.val2.value, hash_size);\n \t\tprintf(\"val 2 %s\", hex);\n \t\thex_format(hex, ref->value.val2.target_value,\n-\t\t\t hash_size(hash_id));\n+\t\t\t hash_size);\n \t\tprintf(\"(T %s)\", hex);\n \t\tbreak;\n \tcase REFTABLE_REF_VAL1:\n-\t\thex_format(hex, ref->value.val1, hash_size(hash_id));\n+\t\thex_format(hex, ref->value.val1, hash_size);\n \t\tprintf(\"val 1 %s\", hex);\n \t\tbreak;\n \tcase REFTABLE_REF_DELETION:\n@@ -282,6 +282,11 @@ void reftable_ref_record_print(const struct reftable_ref_record *ref,\n \tprintf(\"}\\n\");\n }\n \n+void reftable_ref_record_print(const struct reftable_ref_record *ref,\n+\t\t\t uint32_t hash_id) {\n+\treftable_ref_record_print_sz(ref, hash_size(hash_id));\n+}\n+\n static void reftable_ref_record_release_void(void *rec)\n {\n \treftable_ref_record_release(rec);\n@@ -443,6 +448,12 @@ static int reftable_ref_record_equal_void(const void *a,\n \treturn reftable_ref_record_equal(ra, rb, hash_size);\n }\n \n+static void reftable_ref_record_print_void(const void *rec,\n+\t\t\t\t\t int hash_size)\n+{\n+\treftable_ref_record_print_sz((struct reftable_ref_record *) rec, hash_size);\n+}\n+\n static struct reftable_record_vtable reftable_ref_record_vtable = {\n \t.key = &reftable_ref_record_key,\n \t.type = BLOCK_TYPE_REF,\n@@ -453,6 +464,7 @@ static struct reftable_record_vtable reftable_ref_record_vtable = {\n \t.release = &reftable_ref_record_release_void,\n \t.is_deletion = &reftable_ref_record_is_deletion_void,\n \t.equal = &reftable_ref_record_equal_void,\n+\t.print = &reftable_ref_record_print_void,\n };\n \n static void reftable_obj_record_key(const void *r, struct strbuf *dest)\n@@ -471,6 +483,21 @@ static void reftable_obj_record_release(void *rec)\n \tmemset(obj, 0, sizeof(struct reftable_obj_record));\n }\n \n+static void reftable_obj_record_print(const void *rec, int hash_size)\n+{\n+\tconst struct reftable_obj_record *obj = rec;\n+\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 };\n+\tstruct strbuf offset_str = STRBUF_INIT;\n+\tint i;\n+\n+\tfor (i = 0; i < obj->offset_len; i++)\n+\t\tstrbuf_addf(&offset_str, \"%\" PRIu64 \" \", obj->offsets[i]);\n+\thex_format(hex, obj->hash_prefix, obj->hash_prefix_len);\n+\tprintf(\"prefix %s (len %d), offsets [%s]\\n\",\n+\t hex, obj->hash_prefix_len, offset_str.buf);\n+\tstrbuf_release(&offset_str);\n+}\n+\n static void reftable_obj_record_copy_from(void *rec, const void *src_rec,\n \t\t\t\t\t int hash_size)\n {\n@@ -617,31 +644,41 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {\n \t.release = &reftable_obj_record_release,\n \t.is_deletion = ¬_a_deletion,\n \t.equal = &reftable_obj_record_equal_void,\n+\t.print = &reftable_obj_record_print,\n };\n \n-void reftable_log_record_print(struct reftable_log_record *log,\n-\t\t\t uint32_t hash_id)\n+static void reftable_log_record_print_sz(struct reftable_log_record *log,\n+\t\t\t\t\t int hash_size)\n {\n \tchar hex[GIT_MAX_HEXSZ + 1] = { 0 };\n \n \tswitch (log->value_type) {\n \tcase REFTABLE_LOG_DELETION:\n-\t\tprintf(\"log{%s(%\" PRIu64 \") delete\", log->refname,\n+\t\tprintf(\"log{%s(%\" PRIu64 \") delete\\n\", log->refname,\n \t\t log->update_index);\n \t\tbreak;\n \tcase REFTABLE_LOG_UPDATE:\n \t\tprintf(\"log{%s(%\" PRIu64 \") %s <%s> %\" PRIu64 \" %04d\\n\",\n-\t\t log->refname, log->update_index, log->value.update.name,\n-\t\t log->value.update.email, log->value.update.time,\n+\t\t log->refname, log->update_index,\n+\t\t log->value.update.name ? log->value.update.name : \"\",\n+\t\t log->value.update.email ? log->value.update.email : \"\",\n+\t\t log->value.update.time,\n \t\t log->value.update.tz_offset);\n-\t\thex_format(hex, log->value.update.old_hash, hash_size(hash_id));\n+\t\thex_format(hex, log->value.update.old_hash, hash_size);\n \t\tprintf(\"%s => \", hex);\n-\t\thex_format(hex, log->value.update.new_hash, hash_size(hash_id));\n-\t\tprintf(\"%s\\n\\n%s\\n}\\n\", hex, log->value.update.message);\n+\t\thex_format(hex, log->value.update.new_hash, hash_size);\n+\t\tprintf(\"%s\\n\\n%s\\n}\\n\", hex,\n+\t\t log->value.update.message ? log->value.update.message : \"\");\n \t\tbreak;\n \t}\n }\n \n+void reftable_log_record_print(struct reftable_log_record *log,\n+\t\t\t\t uint32_t hash_id)\n+{\n+\treftable_log_record_print_sz(log, hash_size(hash_id));\n+}\n+\n static void reftable_log_record_key(const void *r, struct strbuf *dest)\n {\n \tconst struct reftable_log_record *rec =\n@@ -959,6 +996,11 @@ static int reftable_log_record_is_deletion_void(const void *p)\n \t\t(const struct reftable_log_record *)p);\n }\n \n+static void reftable_log_record_print_void(const void *rec, int hash_size)\n+{\n+\treftable_log_record_print_sz((struct reftable_log_record*)rec, hash_size);\n+}\n+\n static struct reftable_record_vtable reftable_log_record_vtable = {\n \t.key = &reftable_log_record_key,\n \t.type = BLOCK_TYPE_LOG,\n@@ -968,7 +1010,8 @@ static struct reftable_record_vtable reftable_log_record_vtable = {\n \t.decode = &reftable_log_record_decode,\n \t.release = &reftable_log_record_release_void,\n \t.is_deletion = &reftable_log_record_is_deletion_void,\n-\t.equal = &reftable_log_record_equal_void\n+\t.equal = &reftable_log_record_equal_void,\n+\t.print = &reftable_log_record_print_void,\n };\n \n static void reftable_index_record_key(const void *r, struct strbuf *dest)\n@@ -1043,6 +1086,13 @@ static int reftable_index_record_equal(const void *a, const void *b, int hash_si\n \treturn ia->offset == ib->offset && !strbuf_cmp(&ia->last_key, &ib->last_key);\n }\n \n+static void reftable_index_record_print(const void *rec, int hash_size)\n+{\n+\tconst struct reftable_index_record *idx = rec;\n+\t/* TODO: escape null chars? */\n+\tprintf(\"\\\"%s\\\" %\" PRIu64 \"\\n\", idx->last_key.buf, idx->offset);\n+}\n+\n static struct reftable_record_vtable reftable_index_record_vtable = {\n \t.key = &reftable_index_record_key,\n \t.type = BLOCK_TYPE_INDEX,\n@@ -1053,6 +1103,7 @@ static struct reftable_record_vtable reftable_index_record_vtable = {\n \t.release = &reftable_index_record_release,\n \t.is_deletion = ¬_a_deletion,\n \t.equal = &reftable_index_record_equal,\n+\t.print = &reftable_index_record_print,\n };\n \n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)\n@@ -1255,3 +1306,9 @@ struct reftable_record reftable_new_record(uint8_t typ)\n \t}\n \treturn clean;\n }\n+\n+void reftable_record_print(struct reftable_record *rec, int hash_size)\n+{\n+\tprintf(\"'%c': \", rec->type);\n+\treftable_record_vtable(rec)->print(reftable_record_data(rec), hash_size);\n+}\ndiff --git a/reftable/record.h b/reftable/record.h\nindex 010a322e901..fd80cd451d5 100644\n--- a/reftable/record.h\n+++ b/reftable/record.h\n@@ -61,6 +61,9 @@ struct reftable_record_vtable {\n \n \t/* Are two records equal? This assumes they have the same type. Returns 0 for non-equal. */\n \tint (*equal)(const void *a, const void *b, int hash_size);\n+\n+\t/* Print on stdout, for debugging. */\n+\tvoid (*print)(const void *rec, int hash_size);\n };\n \n /* returns true for recognized block types. Block start with the block type. */\n@@ -112,6 +115,7 @@ struct reftable_record {\n \n /* see struct record_vtable */\n int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size);\n+void reftable_record_print(struct reftable_record *rec, int hash_size);\n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest);\n uint8_t reftable_record_type(struct reftable_record *rec);\n void reftable_record_copy_from(struct reftable_record *rec,\ndiff --git a/reftable/record_test.c b/reftable/record_test.c\nindex c6fdd1925a9..f91ea5e8830 100644\n--- a/reftable/record_test.c\n+++ b/reftable/record_test.c\n@@ -25,6 +25,10 @@ static void test_copy(struct reftable_record *rec)\n \t/* do it twice to catch memory leaks */\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \tEXPECT(reftable_record_equal(rec, ©, GIT_SHA1_RAWSZ));\n+\n+\tputs(\"testing print coverage:\\n\");\n+\treftable_record_print(©, GIT_SHA1_RAWSZ);\n+\n \treftable_record_release(©);\n }\n \n@@ -176,7 +180,8 @@ static void test_reftable_log_record_equal(void)\n static void test_reftable_log_record_roundtrip(void)\n {\n \tint i;\n-\tstruct reftable_log_record in[2] = {\n+\n+\tstruct reftable_log_record in[] = {\n \t\t{\n \t\t\t.refname = xstrdup(\"refs/heads/master\"),\n \t\t\t.update_index = 42,\n@@ -197,10 +202,24 @@ static void test_reftable_log_record_roundtrip(void)\n \t\t\t.refname = xstrdup(\"refs/heads/master\"),\n \t\t\t.update_index = 22,\n \t\t\t.value_type = REFTABLE_LOG_DELETION,\n+\t\t},\n+\t\t{\n+\t\t\t.refname = xstrdup(\"branch\"),\n+\t\t\t.update_index = 33,\n+\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n+\t\t\t.value = {\n+\t\t\t\t.update = {\n+\t\t\t\t\t.old_hash = reftable_malloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t.new_hash = reftable_malloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t/* rest of fields left empty. */\n+\t\t\t\t},\n+\t\t\t},\n \t\t}\n \t};\n \tset_test_hash(in[0].value.update.new_hash, 1);\n \tset_test_hash(in[0].value.update.old_hash, 2);\n+\tset_test_hash(in[2].value.update.new_hash, 3);\n+\tset_test_hash(in[2].value.update.old_hash, 4);\n \tfor (i = 0; i < ARRAY_SIZE(in); i++) {\n \t\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG };\n \t\tstruct strbuf key = STRBUF_INIT;\n-- \ngitgitgadget\n\n"},{"id":"444810","messageId":"e16bf0c5212ae85daa0d6aa2c78d551824b542bd.1640199396.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","subject":"[PATCH v5 16/16] reftable: be more paranoid about 0-length memcpy calls","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-22T18:56:36Z","receivedAt":"2021-12-22T18:57:06Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 10 +++++++---\n 1 file changed, 7 insertions(+), 3 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex fbaa1fbef56..423e687b220 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -126,7 +126,8 @@ static int encode_string(char *str, struct string_view s)\n \tstring_view_consume(&s, n);\n \tif (s.len < l)\n \t\treturn -1;\n-\tmemcpy(s.buf, str, l);\n+\tif (l)\n+\t\tmemcpy(s.buf, str, l);\n \tstring_view_consume(&s, l);\n \n \treturn start.len - s.len;\n@@ -153,7 +154,9 @@ int reftable_encode_key(int *restart, struct string_view dest,\n \n \tif (dest.len < suffix_len)\n \t\treturn -1;\n-\tmemcpy(dest.buf, key.buf + prefix_len, suffix_len);\n+\n+\tif (suffix_len)\n+\t\tmemcpy(dest.buf, key.buf + prefix_len, suffix_len);\n \tstring_view_consume(&dest, suffix_len);\n \n \treturn start.len - dest.len;\n@@ -569,7 +572,8 @@ static int reftable_obj_record_decode(void *rec, struct strbuf key,\n \tuint64_t last;\n \tint j;\n \tr->hash_prefix = reftable_malloc(key.len);\n-\tmemcpy(r->hash_prefix, key.buf, key.len);\n+\tif (key.len)\n+\t\tmemcpy(r->hash_prefix, key.buf, key.len);\n \tr->hash_prefix_len = key.len;\n \n \tif (val_type == 0) {\n-- \ngitgitgadget\n"},{"id":"444811","messageId":"ede2e792ab6130ca7e18fe7d6b9c058e5d2439ba.1640199396.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","subject":"[PATCH v5 14/16] reftable: make reftable_record a tagged union","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-22T18:56:34Z","receivedAt":"2021-12-22T18:57:18Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis reduces the amount of glue code, because we don't need a void pointer or\nvtable within the structure.\n\nThe only snag is that reftable_index_record contain a strbuf, so it cannot be\nzero-initialized. To address this, use reftable_new_record() to return fresh\ninstance, given a record type. Since reftable_new_record() doesn't cause heap\nallocation anymore, it should be balanced with reftable_record_release() rather\nthan reftable_record_destroy().\n\nThanks to Peff for the suggestion.\n\nHelped-by: Jeff King \nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/block.c | 2 +-\n reftable/block_test.c | 22 ++---\n reftable/generic.c | 41 ++++----\n reftable/iter.c | 4 +-\n reftable/merged.c | 33 ++++---\n reftable/pq.c | 3 +-\n reftable/pq_test.c | 27 +++---\n reftable/reader.c | 87 +++++++++--------\n reftable/record.c | 212 +++++++++++++++++++----------------------\n reftable/record.h | 42 ++++----\n reftable/record_test.c | 157 +++++++++++++++---------------\n reftable/writer.c | 41 ++++----\n 12 files changed, 334 insertions(+), 337 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 6c8e8705205..2170748c5e9 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -421,7 +421,7 @@ int block_reader_seek(struct block_reader *br, struct block_iter *it,\n done:\n \tstrbuf_release(&key);\n \tstrbuf_release(&next.last_key);\n-\treftable_record_destroy(&rec);\n+\treftable_record_release(&rec);\n \n \treturn err;\n }\ndiff --git a/reftable/block_test.c b/reftable/block_test.c\nindex 4b3ea262dcb..fa2ee092ec0 100644\n--- a/reftable/block_test.c\n+++ b/reftable/block_test.c\n@@ -26,8 +26,9 @@ static void test_block_read_write(void)\n \tstruct block_writer bw = {\n \t\t.last_key = STRBUF_INIT,\n \t};\n-\tstruct reftable_ref_record ref = { NULL };\n-\tstruct reftable_record rec = { NULL };\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t};\n \tint i = 0;\n \tint n;\n \tstruct block_reader br = { 0 };\n@@ -40,7 +41,6 @@ static void test_block_read_write(void)\n \tblock.source = malloc_block_source();\n \tblock_writer_init(&bw, BLOCK_TYPE_REF, block.data, block_size,\n \t\t\t header_off, hash_size(GIT_SHA1_FORMAT_ID));\n-\treftable_record_from_ref(&rec, &ref);\n \n \tfor (i = 0; i < N; i++) {\n \t\tchar name[100];\n@@ -48,14 +48,14 @@ static void test_block_read_write(void)\n \t\tsnprintf(name, sizeof(name), \"branch%02d\", i);\n \t\tmemset(hash, i, sizeof(hash));\n \n-\t\tref.refname = name;\n-\t\tref.value_type = REFTABLE_REF_VAL1;\n-\t\tref.value.val1 = hash;\n+\t\trec.u.ref.refname = name;\n+\t\trec.u.ref.value_type = REFTABLE_REF_VAL1;\n+\t\trec.u.ref.value.val1 = hash;\n \n \t\tnames[i] = xstrdup(name);\n \t\tn = block_writer_add(&bw, &rec);\n-\t\tref.refname = NULL;\n-\t\tref.value_type = REFTABLE_REF_DELETION;\n+\t\trec.u.ref.refname = NULL;\n+\t\trec.u.ref.value_type = REFTABLE_REF_DELETION;\n \t\tEXPECT(n == 0);\n \t}\n \n@@ -74,7 +74,7 @@ static void test_block_read_write(void)\n \t\tif (r > 0) {\n \t\t\tbreak;\n \t\t}\n-\t\tEXPECT_STREQ(names[j], ref.refname);\n+\t\tEXPECT_STREQ(names[j], rec.u.ref.refname);\n \t\tj++;\n \t}\n \n@@ -92,7 +92,7 @@ static void test_block_read_write(void)\n \t\tn = block_iter_next(&it, &rec);\n \t\tEXPECT(n == 0);\n \n-\t\tEXPECT_STREQ(names[i], ref.refname);\n+\t\tEXPECT_STREQ(names[i], rec.u.ref.refname);\n \n \t\twant.len--;\n \t\tn = block_reader_seek(&br, &it, &want);\n@@ -100,7 +100,7 @@ static void test_block_read_write(void)\n \n \t\tn = block_iter_next(&it, &rec);\n \t\tEXPECT(n == 0);\n-\t\tEXPECT_STREQ(names[10 * (i / 10)], ref.refname);\n+\t\tEXPECT_STREQ(names[10 * (i / 10)], rec.u.ref.refname);\n \n \t\tblock_iter_close(&it);\n \t}\ndiff --git a/reftable/generic.c b/reftable/generic.c\nindex 7a8a738d860..b27d152e89a 100644\n--- a/reftable/generic.c\n+++ b/reftable/generic.c\n@@ -7,6 +7,7 @@ https://developers.google.com/open-source/licenses/bsd\n */\n \n #include \"basics.h\"\n+#include \"constants.h\"\n #include \"record.h\"\n #include \"generic.h\"\n #include \"reftable-iterator.h\"\n@@ -15,23 +16,21 @@ https://developers.google.com/open-source/licenses/bsd\n int reftable_table_seek_ref(struct reftable_table *tab,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_REF,\n+\t\t\t\t .u.ref = {\n+\t\t\t\t\t .refname = (char *)name,\n+\t\t\t\t } };\n \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n }\n \n int reftable_table_seek_log(struct reftable_table *tab,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = ~((uint64_t)0),\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n+\t\t\t\t .u.log = {\n+\t\t\t\t\t .refname = (char *)name,\n+\t\t\t\t\t .update_index = ~((uint64_t)0),\n+\t\t\t\t } };\n \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n }\n \n@@ -129,17 +128,25 @@ void reftable_iterator_destroy(struct reftable_iterator *it)\n int reftable_iterator_next_ref(struct reftable_iterator *it,\n \t\t\t struct reftable_ref_record *ref)\n {\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, ref);\n-\treturn iterator_next(it, &rec);\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = *ref,\n+\t};\n+\tint err = iterator_next(it, &rec);\n+\t*ref = rec.u.ref;\n+\treturn err;\n }\n \n int reftable_iterator_next_log(struct reftable_iterator *it,\n \t\t\t struct reftable_log_record *log)\n {\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, log);\n-\treturn iterator_next(it, &rec);\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.u.log = *log,\n+\t};\n+\tint err = iterator_next(it, &rec);\n+\t*log = rec.u.log;\n+\treturn err;\n }\n \n int iterator_next(struct reftable_iterator *it, struct reftable_record *rec)\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex 93d04f735b8..a8d174c0406 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -32,7 +32,7 @@ static int filtering_ref_iterator_next(void *iter_arg,\n \t\t\t\t struct reftable_record *rec)\n {\n \tstruct filtering_ref_iterator *fri = iter_arg;\n-\tstruct reftable_ref_record *ref = rec->data;\n+\tstruct reftable_ref_record *ref = &rec->u.ref;\n \tint err = 0;\n \twhile (1) {\n \t\terr = reftable_iterator_next_ref(&fri->it, ref);\n@@ -127,7 +127,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n static int indexed_table_ref_iter_next(void *p, struct reftable_record *rec)\n {\n \tstruct indexed_table_ref_iter *it = p;\n-\tstruct reftable_ref_record *ref = rec->data;\n+\tstruct reftable_ref_record *ref = &rec->u.ref;\n \n \twhile (1) {\n \t\tint err = block_iter_next(&it->cur, rec);\ndiff --git a/reftable/merged.c b/reftable/merged.c\nindex e5b53da6db3..2a6efa110d5 100644\n--- a/reftable/merged.c\n+++ b/reftable/merged.c\n@@ -30,7 +30,7 @@ static int merged_iter_init(struct merged_iter *mi)\n \n \t\tif (err > 0) {\n \t\t\treftable_iterator_destroy(&mi->stack[i]);\n-\t\t\treftable_record_destroy(&rec);\n+\t\t\treftable_record_release(&rec);\n \t\t} else {\n \t\t\tstruct pq_entry e = {\n \t\t\t\t.rec = rec,\n@@ -57,18 +57,17 @@ static void merged_iter_close(void *p)\n static int merged_iter_advance_nonnull_subiter(struct merged_iter *mi,\n \t\t\t\t\t size_t idx)\n {\n-\tstruct reftable_record rec = reftable_new_record(mi->typ);\n \tstruct pq_entry e = {\n-\t\t.rec = rec,\n+\t\t.rec = reftable_new_record(mi->typ),\n \t\t.index = idx,\n \t};\n-\tint err = iterator_next(&mi->stack[idx], &rec);\n+\tint err = iterator_next(&mi->stack[idx], &e.rec);\n \tif (err < 0)\n \t\treturn err;\n \n \tif (err > 0) {\n \t\treftable_iterator_destroy(&mi->stack[idx]);\n-\t\treftable_record_destroy(&rec);\n+\t\treftable_record_release(&e.rec);\n \t\treturn 0;\n \t}\n \n@@ -126,11 +125,11 @@ static int merged_iter_next_entry(struct merged_iter *mi,\n \t\tif (err < 0) {\n \t\t\treturn err;\n \t\t}\n-\t\treftable_record_destroy(&top.rec);\n+\t\treftable_record_release(&top.rec);\n \t}\n \n \treftable_record_copy_from(rec, &entry.rec, hash_size(mi->hash_id));\n-\treftable_record_destroy(&entry.rec);\n+\treftable_record_release(&entry.rec);\n \tstrbuf_release(&entry_key);\n \treturn 0;\n }\n@@ -290,11 +289,12 @@ int reftable_merged_table_seek_ref(struct reftable_merged_table *mt,\n \t\t\t\t struct reftable_iterator *it,\n \t\t\t\t const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = {\n+\t\t\t.refname = (char *)name,\n+\t\t},\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n \treturn merged_table_seek_record(mt, it, &rec);\n }\n \n@@ -302,12 +302,11 @@ int reftable_merged_table_seek_log_at(struct reftable_merged_table *mt,\n \t\t\t\t struct reftable_iterator *it,\n \t\t\t\t const char *name, uint64_t update_index)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = update_index,\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n+\t\t\t\t .u.log = {\n+\t\t\t\t\t .refname = (char *)name,\n+\t\t\t\t\t .update_index = update_index,\n+\t\t\t\t } };\n \treturn merged_table_seek_record(mt, it, &rec);\n }\n \ndiff --git a/reftable/pq.c b/reftable/pq.c\nindex efc474017a2..96ca6dd37b3 100644\n--- a/reftable/pq.c\n+++ b/reftable/pq.c\n@@ -74,6 +74,7 @@ struct pq_entry merged_iter_pqueue_remove(struct merged_iter_pqueue *pq)\n void merged_iter_pqueue_add(struct merged_iter_pqueue *pq, struct pq_entry e)\n {\n \tint i = 0;\n+\n \tif (pq->len == pq->cap) {\n \t\tpq->cap = 2 * pq->cap + 1;\n \t\tpq->heap = reftable_realloc(pq->heap,\n@@ -98,7 +99,7 @@ void merged_iter_pqueue_release(struct merged_iter_pqueue *pq)\n {\n \tint i = 0;\n \tfor (i = 0; i < pq->len; i++) {\n-\t\treftable_record_destroy(&pq->heap[i].rec);\n+\t\treftable_record_release(&pq->heap[i].rec);\n \t}\n \tFREE_AND_NULL(pq->heap);\n \tpq->len = pq->cap = 0;\ndiff --git a/reftable/pq_test.c b/reftable/pq_test.c\nindex c9bb05e37b7..7de5e886f35 100644\n--- a/reftable/pq_test.c\n+++ b/reftable/pq_test.c\n@@ -31,7 +31,7 @@ static void test_pq(void)\n \tint N = ARRAY_SIZE(names) - 1;\n \n \tstruct merged_iter_pqueue pq = { NULL };\n-\tconst char *last = NULL;\n+\tchar *last = NULL;\n \n \tint i = 0;\n \tfor (i = 0; i < N; i++) {\n@@ -42,12 +42,10 @@ static void test_pq(void)\n \n \ti = 1;\n \tdo {\n-\t\tstruct reftable_record rec =\n-\t\t\treftable_new_record(BLOCK_TYPE_REF);\n-\t\tstruct pq_entry e = { 0 };\n-\n-\t\treftable_record_as_ref(&rec)->refname = names[i];\n-\t\te.rec = rec;\n+\t\tstruct pq_entry e = { .rec = { .type = BLOCK_TYPE_REF,\n+\t\t\t\t\t .u.ref = {\n+\t\t\t\t\t\t .refname = names[i],\n+\t\t\t\t\t } } };\n \t\tmerged_iter_pqueue_add(&pq, e);\n \t\tmerged_iter_pqueue_check(pq);\n \t\ti = (i * 7) % N;\n@@ -55,19 +53,18 @@ static void test_pq(void)\n \n \twhile (!merged_iter_pqueue_is_empty(pq)) {\n \t\tstruct pq_entry e = merged_iter_pqueue_remove(&pq);\n-\t\tstruct reftable_ref_record *ref =\n-\t\t\treftable_record_as_ref(&e.rec);\n-\n+\t\tstruct reftable_record *rec = &e.rec;\n \t\tmerged_iter_pqueue_check(pq);\n \n+\t\tEXPECT(reftable_record_type(rec) == BLOCK_TYPE_REF);\n \t\tif (last) {\n-\t\t\tEXPECT(strcmp(last, ref->refname) < 0);\n+\t\t\tEXPECT(strcmp(last, rec->u.ref.refname) < 0);\n \t\t}\n-\t\tlast = ref->refname;\n-\t\tref->refname = NULL;\n-\t\treftable_free(ref);\n+\t\t// this is names[i], so don't dealloc.\n+\t\tlast = rec->u.ref.refname;\n+\t\trec->u.ref.refname = NULL;\n+\t\treftable_record_release(rec);\n \t}\n-\n \tfor (i = 0; i < N; i++) {\n \t\treftable_free(names[i]);\n \t}\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex c7a32331c76..a537af5f9a8 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -239,8 +239,7 @@ static int table_iter_next_in_block(struct table_iter *ti,\n {\n \tint res = block_iter_next(&ti->bi, rec);\n \tif (res == 0 && reftable_record_type(rec) == BLOCK_TYPE_REF) {\n-\t\t((struct reftable_ref_record *)rec->data)->update_index +=\n-\t\t\tti->r->min_update_index;\n+\t\trec->u.ref.update_index += ti->r->min_update_index;\n \t}\n \n \treturn res;\n@@ -481,7 +480,7 @@ static int reader_seek_linear(struct reftable_reader *r, struct table_iter *ti,\n \n done:\n \tblock_iter_close(&next.bi);\n-\treftable_record_destroy(&rec);\n+\treftable_record_release(&rec);\n \tstrbuf_release(&want_key);\n \tstrbuf_release(&got_key);\n \treturn err;\n@@ -491,34 +490,35 @@ static int reader_seek_indexed(struct reftable_reader *r,\n \t\t\t struct reftable_iterator *it,\n \t\t\t struct reftable_record *rec)\n {\n-\tstruct reftable_index_record want_index = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record want_index_rec = { NULL };\n-\tstruct reftable_index_record index_result = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record index_result_rec = { NULL };\n+\tstruct reftable_record want_index = {\n+\t\t.type = BLOCK_TYPE_INDEX, .u.idx = { .last_key = STRBUF_INIT }\n+\t};\n+\tstruct reftable_record index_result = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = { .last_key = STRBUF_INIT },\n+\t};\n \tstruct table_iter index_iter = TABLE_ITER_INIT;\n \tstruct table_iter next = TABLE_ITER_INIT;\n \tint err = 0;\n \n-\treftable_record_key(rec, &want_index.last_key);\n-\treftable_record_from_index(&want_index_rec, &want_index);\n-\treftable_record_from_index(&index_result_rec, &index_result);\n-\n+\treftable_record_key(rec, &want_index.u.idx.last_key);\n \terr = reader_start(r, &index_iter, reftable_record_type(rec), 1);\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = reader_seek_linear(r, &index_iter, &want_index_rec);\n+\terr = reader_seek_linear(r, &index_iter, &want_index);\n \twhile (1) {\n-\t\terr = table_iter_next(&index_iter, &index_result_rec);\n+\t\terr = table_iter_next(&index_iter, &index_result);\n \t\ttable_iter_block_done(&index_iter);\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = reader_table_iter_at(r, &next, index_result.offset, 0);\n+\t\terr = reader_table_iter_at(r, &next, index_result.u.idx.offset,\n+\t\t\t\t\t 0);\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = block_iter_seek(&next.bi, &want_index.last_key);\n+\t\terr = block_iter_seek(&next.bi, &want_index.u.idx.last_key);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n@@ -546,8 +546,8 @@ static int reader_seek_indexed(struct reftable_reader *r,\n done:\n \tblock_iter_close(&next.bi);\n \ttable_iter_close(&index_iter);\n-\treftable_record_release(&want_index_rec);\n-\treftable_record_release(&index_result_rec);\n+\treftable_record_release(&want_index);\n+\treftable_record_release(&index_result);\n \treturn err;\n }\n \n@@ -596,11 +596,12 @@ static int reader_seek(struct reftable_reader *r, struct reftable_iterator *it,\n int reftable_reader_seek_ref(struct reftable_reader *r,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = {\n+\t\t\t.refname = (char *)name,\n+\t\t},\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n \treturn reader_seek(r, it, &rec);\n }\n \n@@ -608,12 +609,11 @@ int reftable_reader_seek_log_at(struct reftable_reader *r,\n \t\t\t\tstruct reftable_iterator *it, const char *name,\n \t\t\t\tuint64_t update_index)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = update_index,\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n+\t\t\t\t .u.log = {\n+\t\t\t\t\t .refname = (char *)name,\n+\t\t\t\t\t .update_index = update_index,\n+\t\t\t\t } };\n \treturn reader_seek(r, it, &rec);\n }\n \n@@ -657,31 +657,33 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \t\t\t\t\t struct reftable_iterator *it,\n \t\t\t\t\t uint8_t *oid)\n {\n-\tstruct reftable_obj_record want = {\n-\t\t.hash_prefix = oid,\n-\t\t.hash_prefix_len = r->object_id_len,\n+\tstruct reftable_record want = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.u.obj = {\n+\t\t\t.hash_prefix = oid,\n+\t\t\t.hash_prefix_len = r->object_id_len,\n+\t\t},\n \t};\n-\tstruct reftable_record want_rec = { NULL };\n \tstruct reftable_iterator oit = { NULL };\n-\tstruct reftable_obj_record got = { NULL };\n-\tstruct reftable_record got_rec = { NULL };\n+\tstruct reftable_record got = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.u.obj = { 0 },\n+\t};\n \tint err = 0;\n \tstruct indexed_table_ref_iter *itr = NULL;\n \n \t/* Look through the reverse index. */\n-\treftable_record_from_obj(&want_rec, &want);\n-\terr = reader_seek(r, &oit, &want_rec);\n+\terr = reader_seek(r, &oit, &want);\n \tif (err != 0)\n \t\tgoto done;\n \n \t/* read out the reftable_obj_record */\n-\treftable_record_from_obj(&got_rec, &got);\n-\terr = iterator_next(&oit, &got_rec);\n+\terr = iterator_next(&oit, &got);\n \tif (err < 0)\n \t\tgoto done;\n \n-\tif (err > 0 ||\n-\t memcmp(want.hash_prefix, got.hash_prefix, r->object_id_len)) {\n+\tif (err > 0 || memcmp(want.u.obj.hash_prefix, got.u.obj.hash_prefix,\n+\t\t\t r->object_id_len)) {\n \t\t/* didn't find it; return empty iterator */\n \t\titerator_set_empty(it);\n \t\terr = 0;\n@@ -689,15 +691,16 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \t}\n \n \terr = new_indexed_table_ref_iter(&itr, r, oid, hash_size(r->hash_id),\n-\t\t\t\t\t got.offsets, got.offset_len);\n+\t\t\t\t\t got.u.obj.offsets,\n+\t\t\t\t\t got.u.obj.offset_len);\n \tif (err < 0)\n \t\tgoto done;\n-\tgot.offsets = NULL;\n+\tgot.u.obj.offsets = NULL;\n \titerator_from_indexed_table_ref_iter(it, itr);\n \n done:\n \treftable_iterator_destroy(&oit);\n-\treftable_record_release(&got_rec);\n+\treftable_record_release(&got);\n \treturn err;\n }\n \ndiff --git a/reftable/record.c b/reftable/record.c\nindex 2a9e41a992e..a8cee628942 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -15,6 +15,10 @@ https://developers.google.com/open-source/licenses/bsd\n #include \"reftable-error.h\"\n #include \"basics.h\"\n \n+static struct reftable_record_vtable *\n+reftable_record_vtable(struct reftable_record *rec);\n+static void *reftable_record_data(struct reftable_record *rec);\n+\n int get_var_int(uint64_t *dest, struct string_view *in)\n {\n \tint ptr = 0;\n@@ -475,12 +479,14 @@ static void reftable_obj_record_copy_from(void *rec, const void *src_rec,\n \t\t(const struct reftable_obj_record *)src_rec;\n \n \treftable_obj_record_release(obj);\n-\t*obj = *src;\n-\tobj->hash_prefix = reftable_malloc(obj->hash_prefix_len);\n-\tmemcpy(obj->hash_prefix, src->hash_prefix, obj->hash_prefix_len);\n+\tobj->hash_prefix = reftable_malloc(src->hash_prefix_len);\n+\tobj->hash_prefix_len = src->hash_prefix_len;\n+\tif (src->hash_prefix_len)\n+\t\tmemcpy(obj->hash_prefix, src->hash_prefix, obj->hash_prefix_len);\n \n-\tobj->offsets = reftable_malloc(obj->offset_len * sizeof(uint64_t));\n-\tCOPY_ARRAY(obj->offsets, src->offsets, obj->offset_len);\n+\tobj->offsets = reftable_malloc(src->offset_len * sizeof(uint64_t));\n+\tobj->offset_len = src->offset_len;\n+\tCOPY_ARRAY(obj->offsets, src->offsets, src->offset_len);\n }\n \n static uint8_t reftable_obj_record_val_type(const void *rec)\n@@ -965,58 +971,6 @@ static struct reftable_record_vtable reftable_log_record_vtable = {\n \t.equal = &reftable_log_record_equal_void\n };\n \n-struct reftable_record reftable_new_record(uint8_t typ)\n-{\n-\tstruct reftable_record rec = { NULL };\n-\tswitch (typ) {\n-\tcase BLOCK_TYPE_REF: {\n-\t\tstruct reftable_ref_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_ref_record));\n-\t\treftable_record_from_ref(&rec, r);\n-\t\treturn rec;\n-\t}\n-\n-\tcase BLOCK_TYPE_OBJ: {\n-\t\tstruct reftable_obj_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_obj_record));\n-\t\treftable_record_from_obj(&rec, r);\n-\t\treturn rec;\n-\t}\n-\tcase BLOCK_TYPE_LOG: {\n-\t\tstruct reftable_log_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_log_record));\n-\t\treftable_record_from_log(&rec, r);\n-\t\treturn rec;\n-\t}\n-\tcase BLOCK_TYPE_INDEX: {\n-\t\tstruct reftable_index_record empty = { .last_key =\n-\t\t\t\t\t\t\t STRBUF_INIT };\n-\t\tstruct reftable_index_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_index_record));\n-\t\t*r = empty;\n-\t\treftable_record_from_index(&rec, r);\n-\t\treturn rec;\n-\t}\n-\t}\n-\tabort();\n-\treturn rec;\n-}\n-\n-/* clear out the record, yielding the reftable_record data that was\n- * encapsulated. */\n-static void *reftable_record_yield(struct reftable_record *rec)\n-{\n-\tvoid *p = rec->data;\n-\trec->data = NULL;\n-\treturn p;\n-}\n-\n-void reftable_record_destroy(struct reftable_record *rec)\n-{\n-\treftable_record_release(rec);\n-\treftable_free(reftable_record_yield(rec));\n-}\n-\n static void reftable_index_record_key(const void *r, struct strbuf *dest)\n {\n \tconst struct reftable_index_record *rec = r;\n@@ -1103,98 +1057,60 @@ static struct reftable_record_vtable reftable_index_record_vtable = {\n \n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)\n {\n-\trec->ops->key(rec->data, dest);\n+\treftable_record_vtable(rec)->key(reftable_record_data(rec), dest);\n }\n \n uint8_t reftable_record_type(struct reftable_record *rec)\n {\n-\treturn rec->ops->type;\n+\treturn rec->type;\n }\n \n int reftable_record_encode(struct reftable_record *rec, struct string_view dest,\n \t\t\t int hash_size)\n {\n-\treturn rec->ops->encode(rec->data, dest, hash_size);\n+\treturn reftable_record_vtable(rec)->encode(reftable_record_data(rec),\n+\t\t\t\t\t\t dest, hash_size);\n }\n \n void reftable_record_copy_from(struct reftable_record *rec,\n \t\t\t struct reftable_record *src, int hash_size)\n {\n-\tassert(src->ops->type == rec->ops->type);\n+\tassert(src->type == rec->type);\n \n-\trec->ops->copy_from(rec->data, src->data, hash_size);\n+\treftable_record_vtable(rec)->copy_from(reftable_record_data(rec),\n+\t\t\t\t\t reftable_record_data(src),\n+\t\t\t\t\t hash_size);\n }\n \n uint8_t reftable_record_val_type(struct reftable_record *rec)\n {\n-\treturn rec->ops->val_type(rec->data);\n+\treturn reftable_record_vtable(rec)->val_type(reftable_record_data(rec));\n }\n \n int reftable_record_decode(struct reftable_record *rec, struct strbuf key,\n \t\t\t uint8_t extra, struct string_view src, int hash_size)\n {\n-\treturn rec->ops->decode(rec->data, key, extra, src, hash_size);\n+\treturn reftable_record_vtable(rec)->decode(reftable_record_data(rec),\n+\t\t\t\t\t\t key, extra, src, hash_size);\n }\n \n void reftable_record_release(struct reftable_record *rec)\n {\n-\trec->ops->release(rec->data);\n+\treftable_record_vtable(rec)->release(reftable_record_data(rec));\n }\n \n int reftable_record_is_deletion(struct reftable_record *rec)\n {\n-\treturn rec->ops->is_deletion(rec->data);\n+\treturn reftable_record_vtable(rec)->is_deletion(\n+\t\treftable_record_data(rec));\n }\n \n int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size)\n {\n-\tif (a->ops != b->ops)\n+\tif (a->type != b->type)\n \t\treturn 0;\n-\treturn a->ops->equal(a->data, b->data, hash_size);\n-}\n-\n-void reftable_record_from_ref(struct reftable_record *rec,\n-\t\t\t struct reftable_ref_record *ref_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = ref_rec;\n-\trec->ops = &reftable_ref_record_vtable;\n-}\n-\n-void reftable_record_from_obj(struct reftable_record *rec,\n-\t\t\t struct reftable_obj_record *obj_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = obj_rec;\n-\trec->ops = &reftable_obj_record_vtable;\n-}\n-\n-void reftable_record_from_index(struct reftable_record *rec,\n-\t\t\t\tstruct reftable_index_record *index_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = index_rec;\n-\trec->ops = &reftable_index_record_vtable;\n-}\n-\n-void reftable_record_from_log(struct reftable_record *rec,\n-\t\t\t struct reftable_log_record *log_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = log_rec;\n-\trec->ops = &reftable_log_record_vtable;\n-}\n-\n-struct reftable_ref_record *reftable_record_as_ref(struct reftable_record *rec)\n-{\n-\tassert(reftable_record_type(rec) == BLOCK_TYPE_REF);\n-\treturn rec->data;\n-}\n-\n-struct reftable_log_record *reftable_record_as_log(struct reftable_record *rec)\n-{\n-\tassert(reftable_record_type(rec) == BLOCK_TYPE_LOG);\n-\treturn rec->data;\n+\treturn reftable_record_vtable(a)->equal(\n+\t\treftable_record_data(a), reftable_record_data(b), hash_size);\n }\n \n static int hash_equal(uint8_t *a, uint8_t *b, int hash_size)\n@@ -1267,3 +1183,75 @@ void string_view_consume(struct string_view *s, int n)\n \ts->buf += n;\n \ts->len -= n;\n }\n+\n+static void *reftable_record_data(struct reftable_record *rec)\n+{\n+\tswitch (rec->type) {\n+\tcase BLOCK_TYPE_REF:\n+\t\treturn &rec->u.ref;\n+\tcase BLOCK_TYPE_LOG:\n+\t\treturn &rec->u.log;\n+\tcase BLOCK_TYPE_INDEX:\n+\t\treturn &rec->u.idx;\n+\tcase BLOCK_TYPE_OBJ:\n+\t\treturn &rec->u.obj;\n+\t}\n+\tabort();\n+}\n+\n+static struct reftable_record_vtable *\n+reftable_record_vtable(struct reftable_record *rec)\n+{\n+\tswitch (rec->type) {\n+\tcase BLOCK_TYPE_REF:\n+\t\treturn &reftable_ref_record_vtable;\n+\tcase BLOCK_TYPE_LOG:\n+\t\treturn &reftable_log_record_vtable;\n+\tcase BLOCK_TYPE_INDEX:\n+\t\treturn &reftable_index_record_vtable;\n+\tcase BLOCK_TYPE_OBJ:\n+\t\treturn &reftable_obj_record_vtable;\n+\t}\n+\tabort();\n+}\n+\n+struct reftable_record reftable_new_record(uint8_t typ)\n+{\n+\tstruct reftable_record clean = {\n+\t\t.type = typ,\n+\t};\n+\n+\t/* the following is involved, but the naive solution (just return\n+\t * `clean` as is, except for BLOCK_TYPE_INDEX), returns a garbage\n+\t * clean.u.obj.offsets pointer on Windows VS CI. Go figure.\n+\t */\n+\tswitch (typ) {\n+\tcase BLOCK_TYPE_OBJ:\n+\t{\n+\t\tstruct reftable_obj_record obj = { 0 };\n+\t\tclean.u.obj = obj;\n+\t\tbreak;\n+\t}\n+\tcase BLOCK_TYPE_INDEX:\n+\t{\n+\t\tstruct reftable_index_record idx = {\n+\t\t\t.last_key = STRBUF_INIT,\n+\t\t};\n+\t\tclean.u.idx = idx;\n+\t\tbreak;\n+\t}\n+\tcase BLOCK_TYPE_REF:\n+\t{\n+\t\tstruct reftable_ref_record ref = { 0 };\n+\t\tclean.u.ref = ref;\n+\t\tbreak;\n+\t}\n+\tcase BLOCK_TYPE_LOG:\n+\t{\n+\t\tstruct reftable_log_record log = { 0 };\n+\t\tclean.u.log = log;\n+\t\tbreak;\n+\t}\n+\t}\n+\treturn clean;\n+}\ndiff --git a/reftable/record.h b/reftable/record.h\nindex da75d7d1f11..010a322e901 100644\n--- a/reftable/record.h\n+++ b/reftable/record.h\n@@ -63,16 +63,10 @@ struct reftable_record_vtable {\n \tint (*equal)(const void *a, const void *b, int hash_size);\n };\n \n-/* record is a generic wrapper for different types of records. */\n-struct reftable_record {\n-\tvoid *data;\n-\tstruct reftable_record_vtable *ops;\n-};\n-\n /* returns true for recognized block types. Block start with the block type. */\n int reftable_is_block_type(uint8_t typ);\n \n-/* creates a malloced record of the given type. Dispose with record_destroy */\n+/* return an initialized record for the given type */\n struct reftable_record reftable_new_record(uint8_t typ);\n \n /* Encode `key` into `dest`. Sets `is_restart` to indicate a restart. Returns\n@@ -100,6 +94,22 @@ struct reftable_obj_record {\n \tint offset_len;\n };\n \n+/* record is a generic wrapper for different types of records. It is normally\n+ * created on the stack, or embedded within another struct. If the type is\n+ * known, a fresh instance can be initialized explicitly. Otherwise, use\n+ * reftable_new_record() to initialize generically (as the index_record is not\n+ * valid as 0-initialized structure)\n+ */\n+struct reftable_record {\n+\tuint8_t type;\n+\tunion {\n+\t\tstruct reftable_ref_record ref;\n+\t\tstruct reftable_log_record log;\n+\t\tstruct reftable_obj_record obj;\n+\t\tstruct reftable_index_record idx;\n+\t} u;\n+};\n+\n /* see struct record_vtable */\n int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size);\n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest);\n@@ -114,25 +124,9 @@ int reftable_record_decode(struct reftable_record *rec, struct strbuf key,\n \t\t\t int hash_size);\n int reftable_record_is_deletion(struct reftable_record *rec);\n \n-/* zeroes out the embedded record */\n+/* frees and zeroes out the embedded record */\n void reftable_record_release(struct reftable_record *rec);\n \n-/* clear and deallocate embedded record, and zero `rec`. */\n-void reftable_record_destroy(struct reftable_record *rec);\n-\n-/* initialize generic records from concrete records. The generic record should\n- * be zeroed out. */\n-void reftable_record_from_obj(struct reftable_record *rec,\n-\t\t\t struct reftable_obj_record *objrec);\n-void reftable_record_from_index(struct reftable_record *rec,\n-\t\t\t\tstruct reftable_index_record *idxrec);\n-void reftable_record_from_ref(struct reftable_record *rec,\n-\t\t\t struct reftable_ref_record *refrec);\n-void reftable_record_from_log(struct reftable_record *rec,\n-\t\t\t struct reftable_log_record *logrec);\n-struct reftable_ref_record *reftable_record_as_ref(struct reftable_record *ref);\n-struct reftable_log_record *reftable_record_as_log(struct reftable_record *ref);\n-\n /* for qsort. */\n int reftable_ref_record_compare_name(const void *a, const void *b);\n \ndiff --git a/reftable/record_test.c b/reftable/record_test.c\nindex 92680848156..c6fdd1925a9 100644\n--- a/reftable/record_test.c\n+++ b/reftable/record_test.c\n@@ -16,13 +16,16 @@\n \n static void test_copy(struct reftable_record *rec)\n {\n-\tstruct reftable_record copy =\n-\t\treftable_new_record(reftable_record_type(rec));\n+\tstruct reftable_record copy = { 0 };\n+\tuint8_t typ;\n+\n+\ttyp = reftable_record_type(rec);\n+\tcopy = reftable_new_record(typ);\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \t/* do it twice to catch memory leaks */\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \tEXPECT(reftable_record_equal(rec, ©, GIT_SHA1_RAWSZ));\n-\treftable_record_destroy(©);\n+\treftable_record_release(©);\n }\n \n static void test_varint_roundtrip(void)\n@@ -95,61 +98,58 @@ static void test_reftable_ref_record_roundtrip(void)\n \tint i = 0;\n \n \tfor (i = REFTABLE_REF_DELETION; i < REFTABLE_NR_REF_VALUETYPES; i++) {\n-\t\tstruct reftable_ref_record in = { NULL };\n-\t\tstruct reftable_ref_record out = { NULL };\n-\t\tstruct reftable_record rec_out = { NULL };\n+\t\tstruct reftable_record in = {\n+\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t};\n+\t\tstruct reftable_record out = { .type = BLOCK_TYPE_REF };\n \t\tstruct strbuf key = STRBUF_INIT;\n-\t\tstruct reftable_record rec = { NULL };\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n \t\t\t.buf = buffer,\n \t\t\t.len = sizeof(buffer),\n \t\t};\n-\n \t\tint n, m;\n \n-\t\tin.value_type = i;\n+\t\tin.u.ref.value_type = i;\n \t\tswitch (i) {\n \t\tcase REFTABLE_REF_DELETION:\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_VAL1:\n-\t\t\tin.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val1, 1);\n+\t\t\tin.u.ref.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n+\t\t\tset_hash(in.u.ref.value.val1, 1);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_VAL2:\n-\t\t\tin.value.val2.value = reftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val2.value, 1);\n-\t\t\tin.value.val2.target_value =\n+\t\t\tin.u.ref.value.val2.value =\n \t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val2.target_value, 2);\n+\t\t\tset_hash(in.u.ref.value.val2.value, 1);\n+\t\t\tin.u.ref.value.val2.target_value =\n+\t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n+\t\t\tset_hash(in.u.ref.value.val2.target_value, 2);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_SYMREF:\n-\t\t\tin.value.symref = xstrdup(\"target\");\n+\t\t\tin.u.ref.value.symref = xstrdup(\"target\");\n \t\t\tbreak;\n \t\t}\n-\t\tin.refname = xstrdup(\"refs/heads/master\");\n+\t\tin.u.ref.refname = xstrdup(\"refs/heads/master\");\n \n-\t\treftable_record_from_ref(&rec, &in);\n-\t\ttest_copy(&rec);\n+\t\ttest_copy(&in);\n \n-\t\tEXPECT(reftable_record_val_type(&rec) == i);\n+\t\tEXPECT(reftable_record_val_type(&in) == i);\n \n-\t\treftable_record_key(&rec, &key);\n-\t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\t\treftable_record_key(&in, &key);\n+\t\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n > 0);\n \n \t\t/* decode into a non-zero reftable_record to test for leaks. */\n-\n-\t\treftable_record_from_ref(&rec_out, &out);\n-\t\tm = reftable_record_decode(&rec_out, key, i, dest,\n-\t\t\t\t\t GIT_SHA1_RAWSZ);\n+\t\tm = reftable_record_decode(&out, key, i, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_ref_record_equal(&in, &out, GIT_SHA1_RAWSZ));\n-\t\treftable_record_release(&rec_out);\n+\t\tEXPECT(reftable_ref_record_equal(&in.u.ref, &out.u.ref,\n+\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n+\t\treftable_record_release(&in);\n \n \t\tstrbuf_release(&key);\n-\t\treftable_ref_record_release(&in);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n@@ -202,7 +202,7 @@ static void test_reftable_log_record_roundtrip(void)\n \tset_test_hash(in[0].value.update.new_hash, 1);\n \tset_test_hash(in[0].value.update.old_hash, 2);\n \tfor (i = 0; i < ARRAY_SIZE(in); i++) {\n-\t\tstruct reftable_record rec = { NULL };\n+\t\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG };\n \t\tstruct strbuf key = STRBUF_INIT;\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n@@ -210,23 +210,25 @@ static void test_reftable_log_record_roundtrip(void)\n \t\t\t.len = sizeof(buffer),\n \t\t};\n \t\t/* populate out, to check for leaks. */\n-\t\tstruct reftable_log_record out = {\n-\t\t\t.refname = xstrdup(\"old name\"),\n-\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n-\t\t\t.value = {\n-\t\t\t\t.update = {\n-\t\t\t\t\t.new_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n-\t\t\t\t\t.old_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n-\t\t\t\t\t.name = xstrdup(\"old name\"),\n-\t\t\t\t\t.email = xstrdup(\"old@email\"),\n-\t\t\t\t\t.message = xstrdup(\"old message\"),\n+\t\tstruct reftable_record out = {\n+\t\t\t.type = BLOCK_TYPE_LOG,\n+\t\t\t.u.log = {\n+\t\t\t\t.refname = xstrdup(\"old name\"),\n+\t\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n+\t\t\t\t.value = {\n+\t\t\t\t\t.update = {\n+\t\t\t\t\t\t.new_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t\t.old_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t\t.name = xstrdup(\"old name\"),\n+\t\t\t\t\t\t.email = xstrdup(\"old@email\"),\n+\t\t\t\t\t\t.message = xstrdup(\"old message\"),\n+\t\t\t\t\t},\n \t\t\t\t},\n \t\t\t},\n \t\t};\n-\t\tstruct reftable_record rec_out = { NULL };\n \t\tint n, m, valtype;\n \n-\t\treftable_record_from_log(&rec, &in[i]);\n+\t\trec.u.log = in[i];\n \n \t\ttest_copy(&rec);\n \n@@ -234,16 +236,16 @@ static void test_reftable_log_record_roundtrip(void)\n \n \t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n >= 0);\n-\t\treftable_record_from_log(&rec_out, &out);\n \t\tvaltype = reftable_record_val_type(&rec);\n-\t\tm = reftable_record_decode(&rec_out, key, valtype, dest,\n+\t\tm = reftable_record_decode(&out, key, valtype, dest,\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_log_record_equal(&in[i], &out, GIT_SHA1_RAWSZ));\n+\t\tEXPECT(reftable_log_record_equal(&in[i], &out.u.log,\n+\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\treftable_log_record_release(&in[i]);\n \t\tstrbuf_release(&key);\n-\t\treftable_record_release(&rec_out);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n@@ -311,41 +313,43 @@ static void test_reftable_obj_record_roundtrip(void)\n \t\t\t\t\t } };\n \tint i = 0;\n \tfor (i = 0; i < ARRAY_SIZE(recs); i++) {\n-\t\tstruct reftable_obj_record in = recs[i];\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n \t\t\t.buf = buffer,\n \t\t\t.len = sizeof(buffer),\n \t\t};\n-\t\tstruct reftable_record rec = { NULL };\n+\t\tstruct reftable_record in = {\n+\t\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t\t.u.obj = recs[i],\n+\t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n-\t\tstruct reftable_obj_record out = { NULL };\n-\t\tstruct reftable_record rec_out = { NULL };\n+\t\tstruct reftable_record out = { .type = BLOCK_TYPE_OBJ };\n \t\tint n, m;\n \t\tuint8_t extra;\n \n-\t\treftable_record_from_obj(&rec, &in);\n-\t\ttest_copy(&rec);\n-\t\treftable_record_key(&rec, &key);\n-\t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\t\ttest_copy(&in);\n+\t\treftable_record_key(&in, &key);\n+\t\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n > 0);\n-\t\textra = reftable_record_val_type(&rec);\n-\t\treftable_record_from_obj(&rec_out, &out);\n-\t\tm = reftable_record_decode(&rec_out, key, extra, dest,\n+\t\textra = reftable_record_val_type(&in);\n+\t\tm = reftable_record_decode(&out, key, extra, dest,\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_record_equal(&rec, &rec_out, GIT_SHA1_RAWSZ));\n+\t\tEXPECT(reftable_record_equal(&in, &out, GIT_SHA1_RAWSZ));\n \t\tstrbuf_release(&key);\n-\t\treftable_record_release(&rec_out);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n static void test_reftable_index_record_roundtrip(void)\n {\n-\tstruct reftable_index_record in = {\n-\t\t.offset = 42,\n-\t\t.last_key = STRBUF_INIT,\n+\tstruct reftable_record in = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = {\n+\t\t\t.offset = 42,\n+\t\t\t.last_key = STRBUF_INIT,\n+\t\t},\n \t};\n \tuint8_t buffer[1024] = { 0 };\n \tstruct string_view dest = {\n@@ -353,31 +357,30 @@ static void test_reftable_index_record_roundtrip(void)\n \t\t.len = sizeof(buffer),\n \t};\n \tstruct strbuf key = STRBUF_INIT;\n-\tstruct reftable_record rec = { NULL };\n-\tstruct reftable_index_record out = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record out_rec = { NULL };\n+\tstruct reftable_record out = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = { .last_key = STRBUF_INIT },\n+\t};\n \tint n, m;\n \tuint8_t extra;\n \n-\tstrbuf_addstr(&in.last_key, \"refs/heads/master\");\n-\treftable_record_from_index(&rec, &in);\n-\treftable_record_key(&rec, &key);\n-\ttest_copy(&rec);\n+\tstrbuf_addstr(&in.u.idx.last_key, \"refs/heads/master\");\n+\treftable_record_key(&in, &key);\n+\ttest_copy(&in);\n \n-\tEXPECT(0 == strbuf_cmp(&key, &in.last_key));\n-\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\tEXPECT(0 == strbuf_cmp(&key, &in.u.idx.last_key));\n+\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(n > 0);\n \n-\textra = reftable_record_val_type(&rec);\n-\treftable_record_from_index(&out_rec, &out);\n-\tm = reftable_record_decode(&out_rec, key, extra, dest, GIT_SHA1_RAWSZ);\n+\textra = reftable_record_val_type(&in);\n+\tm = reftable_record_decode(&out, key, extra, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(m == n);\n \n-\tEXPECT(reftable_record_equal(&rec, &out_rec, GIT_SHA1_RAWSZ));\n+\tEXPECT(reftable_record_equal(&in, &out, GIT_SHA1_RAWSZ));\n \n-\treftable_record_release(&out_rec);\n+\treftable_record_release(&out);\n \tstrbuf_release(&key);\n-\tstrbuf_release(&in.last_key);\n+\tstrbuf_release(&in.u.idx.last_key);\n }\n \n int record_test_main(int argc, const char *argv[])\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex 9fd24fa93ce..02b74d25c0f 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -253,8 +253,10 @@ done:\n int reftable_writer_add_ref(struct reftable_writer *w,\n \t\t\t struct reftable_ref_record *ref)\n {\n-\tstruct reftable_record rec = { NULL };\n-\tstruct reftable_ref_record copy = *ref;\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = *ref,\n+\t};\n \tint err = 0;\n \n \tif (ref->refname == NULL)\n@@ -263,8 +265,7 @@ int reftable_writer_add_ref(struct reftable_writer *w,\n \t ref->update_index > w->max_update_index)\n \t\treturn REFTABLE_API_ERROR;\n \n-\treftable_record_from_ref(&rec, ©);\n-\tcopy.update_index -= w->min_update_index;\n+\trec.u.ref.update_index -= w->min_update_index;\n \n \terr = writer_add_record(w, &rec);\n \tif (err < 0)\n@@ -303,7 +304,10 @@ int reftable_writer_add_refs(struct reftable_writer *w,\n static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n \t\t\t\t\t struct reftable_log_record *log)\n {\n-\tstruct reftable_record rec = { NULL };\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.u.log = *log,\n+\t};\n \tif (w->block_writer &&\n \t block_writer_type(w->block_writer) == BLOCK_TYPE_REF) {\n \t\tint err = writer_finish_public_section(w);\n@@ -313,8 +317,6 @@ static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n \n \tw->next -= w->pending_padding;\n \tw->pending_padding = 0;\n-\n-\treftable_record_from_log(&rec, log);\n \treturn writer_add_record(w, &rec);\n }\n \n@@ -395,8 +397,10 @@ static int writer_finish_section(struct reftable_writer *w)\n \t\tw->index_len = 0;\n \t\tw->index_cap = 0;\n \t\tfor (i = 0; i < idx_len; i++) {\n-\t\t\tstruct reftable_record rec = { NULL };\n-\t\t\treftable_record_from_index(&rec, idx + i);\n+\t\t\tstruct reftable_record rec = {\n+\t\t\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t\t\t.u.idx = idx[i],\n+\t\t\t};\n \t\t\tif (block_writer_add(w->block_writer, &rec) == 0) {\n \t\t\t\tcontinue;\n \t\t\t}\n@@ -464,17 +468,17 @@ static void write_object_record(void *void_arg, void *key)\n {\n \tstruct write_record_arg *arg = void_arg;\n \tstruct obj_index_tree_node *entry = key;\n-\tstruct reftable_obj_record obj_rec = {\n-\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n-\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n-\t\t.offsets = entry->offsets,\n-\t\t.offset_len = entry->offset_len,\n-\t};\n-\tstruct reftable_record rec = { NULL };\n+\tstruct reftable_record\n+\t\trec = { .type = BLOCK_TYPE_OBJ,\n+\t\t\t.u.obj = {\n+\t\t\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n+\t\t\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n+\t\t\t\t.offsets = entry->offsets,\n+\t\t\t\t.offset_len = entry->offset_len,\n+\t\t\t} };\n \tif (arg->err < 0)\n \t\tgoto done;\n \n-\treftable_record_from_obj(&rec, &obj_rec);\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \tif (arg->err == 0)\n \t\tgoto done;\n@@ -487,7 +491,8 @@ static void write_object_record(void *void_arg, void *key)\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \tif (arg->err == 0)\n \t\tgoto done;\n-\tobj_rec.offset_len = 0;\n+\n+\trec.u.obj.offset_len = 0;\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \n \t/* Should be able to write into a fresh block. */\n-- \ngitgitgadget\n\n"},{"id":"444833","messageId":"xmqq1r24gsph.fsf@gitster.g","threadId":"57047","inReplyTo":"e16bf0c5212ae85daa0d6aa2c78d551824b542bd.1640199396.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 16/16] reftable: be more paranoid about 0-length memcpy calls","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-22T22:50:18Z","receivedAt":"2021-12-22T22:50:25Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Han-Wen Nienhuys via GitGitGadget\" writes:\n\n> From: Han-Wen Nienhuys \n>\n> Signed-off-by: Han-Wen Nienhuys \n> ---\n> reftable/record.c | 10 +++++++---\n> 1 file changed, 7 insertions(+), 3 deletions(-)\n>\n> diff --git a/reftable/record.c b/reftable/record.c\n> index fbaa1fbef56..423e687b220 100644\n> --- a/reftable/record.c\n> +++ b/reftable/record.c\n> @@ -126,7 +126,8 @@ static int encode_string(char *str, struct string_view s)\n> \tstring_view_consume(&s, n);\n> \tif (s.len < l)\n> \t\treturn -1;\n> -\tmemcpy(s.buf, str, l);\n> +\tif (l)\n> +\t\tmemcpy(s.buf, str, l);\n> \tstring_view_consume(&s, l);\n> \n> \treturn start.len - s.len;\n> @@ -153,7 +154,9 @@ int reftable_encode_key(int *restart, struct string_view dest,\n> \n> \tif (dest.len < suffix_len)\n> \t\treturn -1;\n> -\tmemcpy(dest.buf, key.buf + prefix_len, suffix_len);\n> +\n> +\tif (suffix_len)\n> +\t\tmemcpy(dest.buf, key.buf + prefix_len, suffix_len);\n> \tstring_view_consume(&dest, suffix_len);\n> \n> \treturn start.len - dest.len;\n> @@ -569,7 +572,8 @@ static int reftable_obj_record_decode(void *rec, struct strbuf key,\n> \tuint64_t last;\n> \tint j;\n> \tr->hash_prefix = reftable_malloc(key.len);\n> -\tmemcpy(r->hash_prefix, key.buf, key.len);\n> +\tif (key.len)\n> +\t\tmemcpy(r->hash_prefix, key.buf, key.len);\n> \tr->hash_prefix_len = key.len;\n> \n> \tif (val_type == 0) {\n\nI am not sure why any of these are needed.\n\nFor a code path that involves a pair, where ptr is lazily\nallocated only when we have contents to store, i.e. ptr can remain\nNULL until len becomes non-zero, memcpy(dst, ptr, len) can become a\nproblem as the standard requires ptr to be a valid even when len is\n0 (ISO/IEC 9899:1999, 7.21.1 String function conventions, paragraph\n2), but none of these three calls to memcpy() do any such thing.\n\nPuzzled.\n"},{"id":"444834","messageId":"xmqqtuf0fe3r.fsf@gitster.g","threadId":"57047","inReplyTo":"9ab631a3b29addaa54415139e7f60a79a19a6edb.1640199396.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 02/16] reftable: fix resource leak in block.c error path","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-22T22:51:04Z","receivedAt":"2021-12-22T22:51:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Han-Wen Nienhuys via GitGitGadget\" writes:\n\n> diff --git a/reftable/reader.c b/reftable/reader.c\n> index 006709a645a..0d16b098f5e 100644\n> --- a/reftable/reader.c\n> +++ b/reftable/reader.c\n> @@ -290,28 +290,34 @@ int reader_init_block_reader(struct reftable_reader *r, struct block_reader *br,\n> \n> \terr = reader_get_block(r, &block, next_off, guess_block_size);\n> \tif (err < 0)\n> -\t\treturn err;\n> +\t\tgoto done;\n> \n> \tblock_size = extract_block_size(block.data, &block_typ, next_off,\n> \t\t\t\t\tr->version);\n> -\tif (block_size < 0)\n> -\t\treturn block_size;\n> -\n> +\tif (block_size < 0) {\n> +\t\terr = block_size;\n> +\t\tgoto done;\n> +\t}\n> \tif (want_typ != BLOCK_TYPE_ANY && block_typ != want_typ) {\n> -\t\treftable_block_done(&block);\n> -\t\treturn 1;\n> +\t\terr = 1;\n> +\t\tgoto done;\n> \t}\n> \n> \tif (block_size > guess_block_size) {\n> \t\treftable_block_done(&block);\n> \t\terr = reader_get_block(r, &block, next_off, block_size);\n> \t\tif (err < 0) {\n> -\t\t\treturn err;\n> +\t\t\tgoto done;\n> \t\t}\n> \t}\n> \n> -\treturn block_reader_init(br, &block, header_off, r->block_size,\n> -\t\t\t\t hash_size(r->hash_id));\n> +\terr = block_reader_init(br, &block, header_off, r->block_size,\n> +\t\t\t\thash_size(r->hash_id));\n> +done:\n> +\tif (err)\n\nIs the convention for reader_init() different from all other\nfunctions? It makes reader wonder why this is not\n\n\tif (err < 0)\n\neven though it is not wrong per-se (as long as \"zero means success\"\nis a part of the return value convention).\n\n> +\t\treftable_block_done(&block);\n> +\n> +\treturn err;\n> }\n\nThis one is new in this round. All look good, other than that one\ncheck for error return.\n\n> diff --git a/reftable/readwrite_test.c b/reftable/readwrite_test.c\n> index 5f6bcc2f775..6e88182a83a 100644\n> --- a/reftable/readwrite_test.c\n> +++ b/reftable/readwrite_test.c\n> @@ -254,6 +254,71 @@ static void test_log_write_read(void)\n> \treader_close(&rd);\n> }\n> \n> +static void test_log_zlib_corruption(void)\n> +{\n> +\tstruct reftable_write_options opts = {\n> +\t\t.block_size = 256,\n> +\t};\n> +\tstruct reftable_iterator it = { 0 };\n> +\tstruct reftable_reader rd = { 0 };\n> +\tstruct reftable_block_source source = { 0 };\n> +\tstruct strbuf buf = STRBUF_INIT;\n> +\tstruct reftable_writer *w =\n> +\t\treftable_new_writer(&strbuf_add_void, &buf, &opts);\n> +\tconst struct reftable_stats *stats = NULL;\n> +\tuint8_t hash1[GIT_SHA1_RAWSZ] = { 1 };\n> +\tuint8_t hash2[GIT_SHA1_RAWSZ] = { 2 };\n\nWill this code be exercised when compiling with SHA256 support? If\nnot, this is perfectly fine, but otherwise, this needs to be MAX,\nnot SHA1, no?\n\n> +\tchar message[100] = { 0 };\n\nYou're filling this to the sizeof(message)-1, so we can afford to\nleave it uninitialized.\n\n> +\tint err, i, n;\n> +\n> +\tstruct reftable_log_record log = {\n> +\t\t.refname = \"refname\",\n> +\t\t.value_type = REFTABLE_LOG_UPDATE,\n> +\t\t.value = {\n> +\t\t\t.update = {\n> +\t\t\t\t.new_hash = hash1,\n> +\t\t\t\t.old_hash = hash2,\n> +\t\t\t\t.name = \"My Name\",\n> +\t\t\t\t.email = \"myname@invalid\",\n> +\t\t\t\t.message = message,\n> +\t\t\t},\n> +\t\t},\n> +\t};\n> +\n> +\tfor (i = 0; i < sizeof(message)-1; i++)\n\nStyle: SP around \"-\" on both sides.\n\n> +\t\tmessage[i] = (uint8_t)(rand() % 64 + ' ');\n> +\n> +\treftable_writer_set_limits(w, 1, 1);\n"},{"id":"444835","messageId":"xmqqmtksfe3j.fsf@gitster.g","threadId":"57047","inReplyTo":"4175089ec432da921d158fec9ccb1902be710af6.1640199396.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 13/16] reftable: remove outdated file reftable.c","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-22T22:51:12Z","receivedAt":"2021-12-22T22:51:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Han-Wen Nienhuys via GitGitGadget\" writes:\n\n> From: Han-Wen Nienhuys \n>\n> This was renamed to generic.c, but the origin was never removed\n>\n> Signed-off-by: Han-Wen Nienhuys \n> ---\n> reftable/reftable.c | 115 --------------------------------------------\n> 1 file changed, 115 deletions(-)\n> delete mode 100644 reftable/reftable.c\n\nThat's embarrassing for all reviewers of past reftable patches.\n"},{"id":"444836","messageId":"xmqqfsqkfe3b.fsf@gitster.g","threadId":"57047","inReplyTo":"67d858ec59e1c62d5e227e85e117b3cfcb3dda2a.1640199396.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 10/16] reftable: handle null refnames in reftable_ref_record_equal","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-22T22:51:20Z","receivedAt":"2021-12-22T22:51:24Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Han-Wen Nienhuys via GitGitGadget\" writes:\n\n> From: Han-Wen Nienhuys \n>\n> Spotted by Coverity.\n>\n> Signed-off-by: Han-Wen Nienhuys \n> ---\n> reftable/record.c | 8 +++++---\n> 1 file changed, 5 insertions(+), 3 deletions(-)\n>\n> diff --git a/reftable/record.c b/reftable/record.c\n> index 8536bd03aa9..8bbcbff1e69 100644\n> --- a/reftable/record.c\n> +++ b/reftable/record.c\n> @@ -1154,9 +1154,11 @@ int reftable_ref_record_equal(struct reftable_ref_record *a,\n> \t\t\t struct reftable_ref_record *b, int hash_size)\n> {\n> \tassert(hash_size > 0);\n> -\tif (!(0 == strcmp(a->refname, b->refname) &&\n> -\t a->update_index == b->update_index &&\n> -\t a->value_type == b->value_type))\n> +\tif (!null_streq(a->refname, b->refname))\n> +\t\treturn 0;\n> +\n> +\tif (a->update_index != b->update_index ||\n> +\t a->value_type != b->value_type)\n> \t\treturn 0;\n\nThe original assumed that the .refname member of these two records\nare filled, so strcmp() would have segfaulted if they were set to\nNULL, which equates to an empty string. I assume that this patch\nis to fix that and change nothing else?\n\nThe original said: we'll make an early return with 0 unless\nall three conditions hold true; (1) names are the same, (2)\nupdate_index members are the same, and (3) value_type members\nare the same.\n\nWe now say, \"we return early with 0 if names are different\", and \"we\nalso return early with 0 if update-index or value-type members are\ndifferent\".\n\nI just found the splitting into two separate statements looked\ndifferent from the original, but they do mean the same, so it is OK.\n\nIn fact, I think the !(A && B && C) is much less readable than\n(!A || !B || !C) in this case, so I am OK with the new version.\n\nLooking good. Thanks.\n\n"},{"id":"444837","messageId":"xmqq8rwcfe36.fsf@gitster.g","threadId":"57047","inReplyTo":"e7f1be7bbec0427d5c10453631c3486936ed2bf0.1640199396.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 01/16] reftable: fix OOB stack write in print functions","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-22T22:51:25Z","receivedAt":"2021-12-22T22:51:29Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Han-Wen Nienhuys via GitGitGadget\" writes:\n\n> From: Han-Wen Nienhuys \n>\n> Signed-off-by: Han-Wen Nienhuys \n> ---\n> reftable/record.c | 4 ++--\n> 1 file changed, 2 insertions(+), 2 deletions(-)\n>\n> diff --git a/reftable/record.c b/reftable/record.c\n> index 6a5dac32dc6..8536bd03aa9 100644\n> --- a/reftable/record.c\n> +++ b/reftable/record.c\n> @@ -254,7 +254,7 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)\n> void reftable_ref_record_print(struct reftable_ref_record *ref,\n> \t\t\t uint32_t hash_id)\n> {\n> -\tchar hex[2 * GIT_SHA256_RAWSZ + 1] = { 0 }; /* BUG */\n> +\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */\n> \tprintf(\"ref{%s(%\" PRIu64 \") \", ref->refname, ref->update_index);\n> \tswitch (ref->value_type) {\n> \tcase REFTABLE_REF_SYMREF:\n> @@ -586,7 +586,7 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {\n> void reftable_log_record_print(struct reftable_log_record *log,\n> \t\t\t uint32_t hash_id)\n> {\n> -\tchar hex[GIT_SHA256_RAWSZ + 1] = { 0 };\n> +\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 };\n> \n> \tswitch (log->value_type) {\n> \tcase REFTABLE_LOG_DELETION:\n\nWow. The first one is future-proofing in case we learn to use even\nlarger hash, but the latter is an outright bug that makes reviewers\nfeel embarrassed, isn't it?\n\nThanks for noticing and fixing it.\n\n"},{"id":"444838","messageId":"xmqq1r24fe31.fsf@gitster.g","threadId":"57047","inReplyTo":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 00/16] Reftable coverity fixes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-22T22:51:30Z","receivedAt":"2021-12-22T22:51:36Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Han-Wen Nienhuys via GitGitGadget\" writes:\n\n> This series was targeted to 'next'.\n>\n> This addresses some complaints from Coverity that Peff reported.\n>\n> v5:\n>\n> * add generic record equality\n> * add generic record printing\n> * const correctness for record-reftable.h\n> * fix windows crash.\n> * 0-length memcpy paranoia\n> * drop unused file.\n\nThanks. Will replace.\n"},{"id":"444855","messageId":"af378ac5-a24c-5110-04a7-3257a909815b@web.de","threadId":"57047","inReplyTo":"xmqq1r24gsph.fsf@gitster.g","subject":"Re: [PATCH v5 16/16] reftable: be more paranoid about 0-length memcpy calls","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2021-12-23T09:49:25Z","receivedAt":"2021-12-23T09:49:48Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 22.12.21 um 23:50 schrieb Junio C Hamano:\n> \"Han-Wen Nienhuys via GitGitGadget\" writes:\n>\n>> From: Han-Wen Nienhuys \n>>\n>> Signed-off-by: Han-Wen Nienhuys \n>> ---\n>> reftable/record.c | 10 +++++++---\n>> 1 file changed, 7 insertions(+), 3 deletions(-)\n>>\n>> diff --git a/reftable/record.c b/reftable/record.c\n>> index fbaa1fbef56..423e687b220 100644\n>> --- a/reftable/record.c\n>> +++ b/reftable/record.c\n>> @@ -126,7 +126,8 @@ static int encode_string(char *str, struct string_view s)\n>> \tstring_view_consume(&s, n);\n>> \tif (s.len < l)\n>> \t\treturn -1;\n>> -\tmemcpy(s.buf, str, l);\n>> +\tif (l)\n>> +\t\tmemcpy(s.buf, str, l);\n>> \tstring_view_consume(&s, l);\n>>\n>> \treturn start.len - s.len;\n>> @@ -153,7 +154,9 @@ int reftable_encode_key(int *restart, struct string_view dest,\n>>\n>> \tif (dest.len < suffix_len)\n>> \t\treturn -1;\n>> -\tmemcpy(dest.buf, key.buf + prefix_len, suffix_len);\n>> +\n>> +\tif (suffix_len)\n>> +\t\tmemcpy(dest.buf, key.buf + prefix_len, suffix_len);\n>> \tstring_view_consume(&dest, suffix_len);\n>>\n>> \treturn start.len - dest.len;\n>> @@ -569,7 +572,8 @@ static int reftable_obj_record_decode(void *rec, struct strbuf key,\n>> \tuint64_t last;\n>> \tint j;\n>> \tr->hash_prefix = reftable_malloc(key.len);\n>> -\tmemcpy(r->hash_prefix, key.buf, key.len);\n>> +\tif (key.len)\n>> +\t\tmemcpy(r->hash_prefix, key.buf, key.len);\n>> \tr->hash_prefix_len = key.len;\n>>\n>> \tif (val_type == 0) {\n>\n> I am not sure why any of these are needed.\n>\n> For a code path that involves a pair, where ptr is lazily\n> allocated only when we have contents to store, i.e. ptr can remain\n> NULL until len becomes non-zero, memcpy(dst, ptr, len) can become a\n> problem as the standard requires ptr to be a valid even when len is\n> 0 (ISO/IEC 9899:1999, 7.21.1 String function conventions, paragraph\n> 2), but none of these three calls to memcpy() do any such thing.\n>\n> Puzzled.\n\nI don't know about the first two, but in the third case dst (i.e.\nr->hash_prefix) might be NULL if key.len == 0, reftable_malloc is malloc\n(which it is, because reftable_set_alloc is never called) and malloc(0)\nreturns NULL (which it might do according to\nhttps://www.man7.org/linux/man-pages/man3/malloc.3.html).\n\nmalloc can return NULL on failure, too, of course, and none of the\nreftable_malloc callers check for that. That seems a bit too\noptimistic.\n\nRené\n"},{"id":"444860","messageId":"CAFQ2z_OXPw9-hLsaD+E1nGESS7eJH5o5sRrr_R_vW1Oa-vvZoA@mail.gmail.com","threadId":"57047","inReplyTo":"xmqq1r24gsph.fsf@gitster.g","subject":"Re: [PATCH v5 16/16] reftable: be more paranoid about 0-length memcpy calls","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-12-23T15:58:04Z","receivedAt":"2021-12-23T15:58:19Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Wed, Dec 22, 2021 at 11:50 PM Junio C Hamano wrote:\n\n> > - memcpy(r->hash_prefix, key.buf, key.len);\n> > + if (key.len)\n> > + memcpy(r->hash_prefix, key.buf, key.len);\n> > r->hash_prefix_len = key.len;\n> >\n> > if (val_type == 0) {\n>\n> I am not sure why any of these are needed.\n\nI'm not sure they are needed, but IMO it's not worth spending brain\ncycles on deciding either way. Checking the length is always a safe\nalternative.\n\nI would support having a safe_memcpy() that does this check centrally\n(note how our array_copy() array function also does this check, even\nif it's not always needed.)\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\nRegistergericht und -nummer: Hamburg, HRB 86891\nSitz der Gesellschaft: Hamburg\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"444861","messageId":"CAFQ2z_MsQjrucS5XiR7sDWb=sZKu0KFYVBspL_dvyq2DoDufKw@mail.gmail.com","threadId":"57047","inReplyTo":"xmqq8rwcfe36.fsf@gitster.g","subject":"Re: [PATCH v5 01/16] reftable: fix OOB stack write in print functions","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-12-23T15:58:48Z","receivedAt":"2021-12-23T15:59:02Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Wed, Dec 22, 2021 at 11:51 PM Junio C Hamano wrote:>\n> Wow. The first one is future-proofing in case we learn to use even\n> larger hash, but the latter is an outright bug that makes reviewers\n> feel embarrassed, isn't it?\n>\n> Thanks for noticing and fixing it.\n\nYou should thank Coverity :-)\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"444862","messageId":"CAFQ2z_OLCzOYXgXCTXyLOwwk7EBkPzwH=KASDmuJbur=q7L1Jg@mail.gmail.com","threadId":"57047","inReplyTo":"xmqqtuf0fe3r.fsf@gitster.g","subject":"Re: [PATCH v5 02/16] reftable: fix resource leak in block.c error path","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-12-23T17:04:12Z","receivedAt":"2021-12-23T17:04:26Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Wed, Dec 22, 2021 at 11:51 PM Junio C Hamano wrote:\n> > + if (err)\n>\n> Is the convention for reader_init() different from all other\n> functions? It makes reader wonder why this is not\n>\n> if (err < 0)\n> > + reftable_block_done(&block);\n>\n> even though it is not wrong per-se (as long as \"zero means success\"\n> is a part of the return value convention).\n\nerr > 0 is returned when we reach the end of the iteration, and this\nfunction can generate err==1.\n\nNormally, block_reader_init() transfers the block to the block_reader.\nIf err > 0, we skip that, so we'd be leaking the block.\n\nAt the same time, this means the \"if (err)\" is superfluous. In the\nsuccess case, the block was transferred to the block_reader, so the\nreftable_block_done() call is a nop.\n\n>\n> > +\n> > + return err;\n> > }\n>\n> This one is new in this round. All look good, other than that one\n> check for error return.\n>\n> > diff --git a/reftable/readwrite_test.c b/reftable/readwrite_test.c\n> > index 5f6bcc2f775..6e88182a83a 100644\n> > --- a/reftable/readwrite_test.c\n> > +++ b/reftable/readwrite_test.c\n> > @@ -254,6 +254,71 @@ static void test_log_write_read(void)\n> > reader_close(&rd);\n> > }\n> >\n> > +static void test_log_zlib_corruption(void)\n> > +{\n> > + struct reftable_write_options opts = {\n> > + .block_size = 256,\n> > + };\n> > + struct reftable_iterator it = { 0 };\n> > + struct reftable_reader rd = { 0 };\n> > + struct reftable_block_source source = { 0 };\n> > + struct strbuf buf = STRBUF_INIT;\n> > + struct reftable_writer *w =\n> > + reftable_new_writer(&strbuf_add_void, &buf, &opts);\n> > + const struct reftable_stats *stats = NULL;\n> > + uint8_t hash1[GIT_SHA1_RAWSZ] = { 1 };\n> > + uint8_t hash2[GIT_SHA1_RAWSZ] = { 2 };\n>\n> Will this code be exercised when compiling with SHA256 support? If\n> not, this is perfectly fine, but otherwise, this needs to be MAX,\n> not SHA1, no?\n\nThe code is parameterized in terms of hash_size, so we don't have to\ntest both flavors exhaustively. There is a\ntest_table_read_write_seek_linear_sha256() that ensures that the basic\nfunctionality works for SHA256.\n\n> > + char message[100] = { 0 };\n>\n> You're filling this to the sizeof(message)-1, so we can afford to\n> leave it uninitialized.\n\nAt the same time, we can afford to initialize it :-)\n\nI'd rather not think about this, and always initialize everything.\n\n> > + for (i = 0; i < sizeof(message)-1; i++)\n>\n> Style: SP around \"-\" on both sides.\n\ndone.\n\n(I assume I don't have to resend the whole series for these small\ntweaks? I'll wait if anyone else has comments, and send a reroll early\nJanuary)\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"444865","messageId":"CAFQ2z_NPqRseabyFynnNe_0Yh9rYeL9js7B-ddL3YjxSXyJeuw@mail.gmail.com","threadId":"57047","inReplyTo":"569fded7-48d7-6f48-47fb-b78c403214fc@gmail.com","subject":"Re: [PATCH 10/10] reftable: make reftable_record a tagged union","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-12-23T17:11:16Z","receivedAt":"2021-12-23T17:11:36Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Wed, Dec 8, 2021 at 7:17 PM Derrick Stolee wrote:\n> Some of these changes are definitely going to have a lot of lines,\n> but when it's the same kind of change it is easy to verify. I'm\n> getting lost in this diff because it's doing too many things at once.\n\nI took some of your advice in the last version of the patch. PTAL.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"444873","messageId":"xmqqo857b11h.fsf@gitster.g","threadId":"57047","inReplyTo":"af378ac5-a24c-5110-04a7-3257a909815b@web.de","subject":"Re: [PATCH v5 16/16] reftable: be more paranoid about 0-length memcpy calls","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-23T18:59:06Z","receivedAt":"2021-12-23T18:59:10Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"René Scharfe writes:\n\n>>> @@ -569,7 +572,8 @@ static int reftable_obj_record_decode(void *rec, struct strbuf key,\n>>> \tuint64_t last;\n>>> \tint j;\n>>> \tr->hash_prefix = reftable_malloc(key.len);\n>>> -\tmemcpy(r->hash_prefix, key.buf, key.len);\n>>> +\tif (key.len)\n>>> +\t\tmemcpy(r->hash_prefix, key.buf, key.len);\n>>> \tr->hash_prefix_len = key.len;\n>>>\n>>> \tif (val_type == 0) {\n>>\n>> I am not sure why any of these are needed.\n>> ...\n> I don't know about the first two, but in the third case dst (i.e.\n> r->hash_prefix) might be NULL if key.len == 0, reftable_malloc is malloc\n> (which it is, because reftable_set_alloc is never called) and malloc(0)\n> returns NULL (which it might do according to\n> https://www.man7.org/linux/man-pages/man3/malloc.3.html).\n>\n> malloc can return NULL on failure, too, of course, and none of the\n> reftable_malloc callers check for that. That seems a bit too\n> optimistic.\n\nYeah, I agree that the real bug in this code is that the returned\nvalue of malloc() is not checked. But checking if key.len is zero\nbefore using memcpy() would not help fix it, or hide it. So I am\nnot sure if that is a counter-argument against \"this check is\npointless\".\n\nThanks.\n\n"},{"id":"444895","messageId":"xmqq4k6y63j7.fsf@gitster.g","threadId":"57047","inReplyTo":"CAFQ2z_OLCzOYXgXCTXyLOwwk7EBkPzwH=KASDmuJbur=q7L1Jg@mail.gmail.com","subject":"Re: [PATCH v5 02/16] reftable: fix resource leak in block.c error path","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-24T04:16:28Z","receivedAt":"2021-12-24T04:16:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Han-Wen Nienhuys writes:\n\n> Normally, block_reader_init() transfers the block to the block_reader.\n> If err > 0, we skip that, so we'd be leaking the block.\n\nThanks. I was about to say that \n\n wants to say more than \"initializes a block\n reader.\"; perhaps \"returns the number of blocks\" or something\n needs to be added.\n\nbut I do not think the lack of the documentation was the source of\nmy confusion.\n\n> At the same time, this means the \"if (err)\" is superfluous. In the\n> success case, the block was transferred to the block_reader, so the\n> reftable_block_done() call is a nop.\n\nI agree that having the \"if (err)\" there is highly misleading. \"if\n(err < 0 || 0 < err)\" would be the more faithful expression of what\nyou explained, and if it were written that way, I wouldn't have been\nas confused as I was.\n\nIn any case, if _done() becomes a safe and quick no-op when 0 block\nwas transferred, losing the conditional would be the way to make the\nresulting code the most readable, I think.\n\n>> > + uint8_t hash1[GIT_SHA1_RAWSZ] = { 1 };\n>> > + uint8_t hash2[GIT_SHA1_RAWSZ] = { 2 };\n>>\n>> Will this code be exercised when compiling with SHA256 support? If\n>> not, this is perfectly fine, but otherwise, this needs to be MAX,\n>> not SHA1, no?\n>\n> The code is parameterized in terms of hash_size, so we don't have to\n> test both flavors exhaustively.\n\nIf it is safe for this function to assume that it will be called\nonly with hash_size of SHA1, then what you wrote is perfectly fine,\nand I agree that there is no particular reason why this test must be\nrepeated for all available hashes.\n\nI just don't know if we have a mechanism to prevent clueless code\nchurners from saying \"let's test both flavors exhaustively\". For\nexample, I see nothing in this function, other than the size of\nthese two hashes, that says that this test function will be compiled\nwith reftable library that expects that the hash function is SHA1.\nHow does this function protect itself from being run in a CI job\nthat uses GIT_TEST_DEFAULT_HASH=sha256, for example? It was why I\nasked the question. It wasn't that I necessarily thought we need to\ntest all combinations (but there is no need not to, if it takes more\nengineering time to exclude it when testing Git as a whole in sha256\nmode).\n\n>> > + char message[100] = { 0 };\n>>\n>> You're filling this to the sizeof(message)-1, so we can afford to\n>> leave it uninitialized.\n>\n> At the same time, we can afford to initialize it :-)\n>\n> I'd rather not think about this, and always initialize everything.\n\nI do not care too deeply in this test-only function, but as a\ngeneral principle, primarily to help less experienced developers who\nmay be reading from the sidelines to avoid copying such an attitude,\nI have to make a note here.\n\nIf you know you will have to fill an array with, or assign to a\nvariable, meaningful value(s), leaving the array or variable\nuninitialized at the declaration time is a much better thing to do\nthan initializing them with less meaningful value(s). It will help\ncompilers and tools to detect a lack of proper assignment and use of\nuninitialized variable (iow, you may know you will have to fill, but\nin the code to do so, your implementation may be botched). Once you\ninitialize at the declaration with \"less meaningful\" value (like\nzero initialization), the tools won't be able to tell when the code\nuses that variable \"uninitialized\" (because the assignment was\nskipped by a bug), since it appears to always be initialied to them.\n\nHelping the tools help us is what those of us, who would rather not\nthink about it, should be doing.\n"},{"id":"444896","messageId":"xmqqy24a4oyq.fsf@gitster.g","threadId":"57047","inReplyTo":"CAFQ2z_OXPw9-hLsaD+E1nGESS7eJH5o5sRrr_R_vW1Oa-vvZoA@mail.gmail.com","subject":"Re: [PATCH v5 16/16] reftable: be more paranoid about 0-length memcpy calls","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-24T04:16:29Z","receivedAt":"2021-12-24T04:16:37Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Han-Wen Nienhuys writes:\n\n> On Wed, Dec 22, 2021 at 11:50 PM Junio C Hamano wrote:\n>\n>> > - memcpy(r->hash_prefix, key.buf, key.len);\n>> > + if (key.len)\n>> > + memcpy(r->hash_prefix, key.buf, key.len);\n>> > r->hash_prefix_len = key.len;\n>> >\n>> > if (val_type == 0) {\n>>\n>> I am not sure why any of these are needed.\n>\n> I'm not sure they are needed, but IMO it's not worth spending brain\n> cycles on deciding either way. Checking the length is always a safe\n> alternative.\n>\n> I would support having a safe_memcpy() that does this check centrally\n> (note how our array_copy() array function also does this check, even\n> if it's not always needed.)\n\nBut your safe_memcpy() should not be\n\n safe_memcpy(dst, src, n)\n {\n\tif (n)\n\t\tmemcpy(dst, src, n);\n\treturn dst;\n }\n\nUsing memcpy() with size==0 is not a crime wrt the language.\nPassing an invalid pointer while doing so is.\n\nIt should rather be\n\n safe_memcpy(dst, src, n)\n {\n\tif (dst)\n\t\tmemcpy(dst, src, n);\n\telse if (n)\n \tBUG(...);\n\treturn dst;\n }\n\nto support a common pattern of that lazily allocates the\nptr only when len goes more than zero from triggering an error from\npicky runtime, compiler and tools. We want to allow \"dst==NULL &&\nn==0\", while still catching \"dst==NULL && n!=0\" as an error.\n\nAnd these places, I do not think use of safe_memcpy() is relevant.\n"},{"id":"444906","messageId":"211224.86v8zegcro.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"xmqqmtksfe3j.fsf@gitster.g","subject":"Re: [PATCH v5 13/16] reftable: remove outdated file reftable.c","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-12-24T16:53:30Z","receivedAt":"2021-12-24T17:01:29Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Dec 22 2021, Junio C Hamano wrote:\n\n> \"Han-Wen Nienhuys via GitGitGadget\" writes:\n>\n>> From: Han-Wen Nienhuys \n>>\n>> This was renamed to generic.c, but the origin was never removed\n>>\n>> Signed-off-by: Han-Wen Nienhuys \n>> ---\n>> reftable/reftable.c | 115 --------------------------------------------\n>> 1 file changed, 115 deletions(-)\n>> delete mode 100644 reftable/reftable.c\n>\n> That's embarrassing for all reviewers of past reftable patches.\n\nAs one of those reviewers: I think it suggests something different:\n\nVarious people looked it over, but this amount of code is just too much\nfor someone to review in one sitting, so it's not unexpected that\nvarious things like this slipped through.\n\nThe current landing of this topic on \"master\" is based on a plan I\nsuggested, i.e. to have the library and its own tests in \"master\" first,\nand not to do any of the integration yet.\n\nExactly because we expected that we would not be getting any of this\nright the first time around.\n\nSo if anything I think these current topics and recent reftable activity\nsuggest that we should have been more willing to get it into \"master\"\nearlier, and not wait until all bugs in it were solved beforehand.\n\nI.e. there's bugs in the code, but they're well-contained, since nothing\nin-tree uses it except its own tests.\n\nAnd by having merged it down we're benefiting from more eyes, both in\nterms of human review, and in the review of various tooling (like\ncoverity) that's being run on \"master\" but not on some random branch\nthat's languishing in \"seen\" for months.\n"},{"id":"444971","messageId":"e288fad3-8a83-448c-d701-47c56bc258e3@web.de","threadId":"57047","inReplyTo":"xmqqo857b11h.fsf@gitster.g","subject":"Re: [PATCH v5 16/16] reftable: be more paranoid about 0-length memcpy calls","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2021-12-26T20:51:15Z","receivedAt":"2021-12-26T20:51:41Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 23.12.21 um 19:59 schrieb Junio C Hamano:\n> René Scharfe writes:\n>\n>>>> @@ -569,7 +572,8 @@ static int reftable_obj_record_decode(void *rec, struct strbuf key,\n>>>> \tuint64_t last;\n>>>> \tint j;\n>>>> \tr->hash_prefix = reftable_malloc(key.len);\n>>>> -\tmemcpy(r->hash_prefix, key.buf, key.len);\n>>>> +\tif (key.len)\n>>>> +\t\tmemcpy(r->hash_prefix, key.buf, key.len);\n>>>> \tr->hash_prefix_len = key.len;\n>>>>\n>>>> \tif (val_type == 0) {\n>>>\n>>> I am not sure why any of these are needed.\n>>> ...\n>> I don't know about the first two, but in the third case dst (i.e.\n>> r->hash_prefix) might be NULL if key.len == 0, reftable_malloc is malloc\n>> (which it is, because reftable_set_alloc is never called) and malloc(0)\n>> returns NULL (which it might do according to\n>> https://www.man7.org/linux/man-pages/man3/malloc.3.html).\n>>\n>> malloc can return NULL on failure, too, of course, and none of the\n>> reftable_malloc callers check for that. That seems a bit too\n>> optimistic.\n>\n> Yeah, I agree that the real bug in this code is that the returned\n> value of malloc() is not checked. But checking if key.len is zero\n> before using memcpy() would not help fix it, or hide it. So I am\n> not sure if that is a counter-argument against \"this check is\n> pointless\".\n\nIt's not -- I was just trying to find a scenario which would cause a\nCoverity warning that could be silenced by such a zero check.\n\nWe can use xmalloc and xrealloc to handle allocation failures and to get\nvalid pointers for zero-sized allocations, like in the patch below.\n\nI don't understand why reftable_set_alloc exists, though -- is there\nreally a use case that requires changing the allocator at runtime?\n\n\ndiff --git a/reftable/publicbasics.c b/reftable/publicbasics.c\nindex 0ad7d5c0ff..438a1b0a7d 100644\n--- a/reftable/publicbasics.c\n+++ b/reftable/publicbasics.c\n@@ -19,14 +19,14 @@ void *reftable_malloc(size_t sz)\n {\n \tif (reftable_malloc_ptr)\n \t\treturn (*reftable_malloc_ptr)(sz);\n-\treturn malloc(sz);\n+\treturn xmalloc(sz);\n }\n\n void *reftable_realloc(void *p, size_t sz)\n {\n \tif (reftable_realloc_ptr)\n \t\treturn (*reftable_realloc_ptr)(p, sz);\n-\treturn realloc(p, sz);\n+\treturn xrealloc(p, sz);\n }\n\n void reftable_free(void *p)\n"},{"id":"444972","messageId":"211226.86pmpjf4xz.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"e288fad3-8a83-448c-d701-47c56bc258e3@web.de","subject":"Re: [PATCH v5 16/16] reftable: be more paranoid about 0-length memcpy calls","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-12-26T21:07:38Z","receivedAt":"2021-12-26T21:10:21Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Sun, Dec 26 2021, René Scharfe wrote:\n\n> Am 23.12.21 um 19:59 schrieb Junio C Hamano:\n>> René Scharfe writes:\n>>\n>>>>> @@ -569,7 +572,8 @@ static int reftable_obj_record_decode(void *rec, struct strbuf key,\n>>>>> \tuint64_t last;\n>>>>> \tint j;\n>>>>> \tr->hash_prefix = reftable_malloc(key.len);\n>>>>> -\tmemcpy(r->hash_prefix, key.buf, key.len);\n>>>>> +\tif (key.len)\n>>>>> +\t\tmemcpy(r->hash_prefix, key.buf, key.len);\n>>>>> \tr->hash_prefix_len = key.len;\n>>>>>\n>>>>> \tif (val_type == 0) {\n>>>>\n>>>> I am not sure why any of these are needed.\n>>>> ...\n>>> I don't know about the first two, but in the third case dst (i.e.\n>>> r->hash_prefix) might be NULL if key.len == 0, reftable_malloc is malloc\n>>> (which it is, because reftable_set_alloc is never called) and malloc(0)\n>>> returns NULL (which it might do according to\n>>> https://www.man7.org/linux/man-pages/man3/malloc.3.html).\n>>>\n>>> malloc can return NULL on failure, too, of course, and none of the\n>>> reftable_malloc callers check for that. That seems a bit too\n>>> optimistic.\n>>\n>> Yeah, I agree that the real bug in this code is that the returned\n>> value of malloc() is not checked. But checking if key.len is zero\n>> before using memcpy() would not help fix it, or hide it. So I am\n>> not sure if that is a counter-argument against \"this check is\n>> pointless\".\n>\n> It's not -- I was just trying to find a scenario which would cause a\n> Coverity warning that could be silenced by such a zero check.\n>\n> We can use xmalloc and xrealloc to handle allocation failures and to get\n> valid pointers for zero-sized allocations, like in the patch below.\n>\n> I don't understand why reftable_set_alloc exists, though -- is there\n> really a use case that requires changing the allocator at runtime?\n\nI don't know why it's there, but I suppose it's for the same reason as\nPCREv2's integration, whose rabbit hole starts at 513f2b0bbd4 (grep:\nmake PCRE2 aware of custom allocator, 2019-10-16).\n\nI.e. for building it as part of git.git we can just replace\nmalloc()/free(), but for other uses as a general linkable library you'd\nwant to replace it at runtime.\n\n"},{"id":"445985","messageId":"CAFQ2z_PDfreeGttTbNTbnr7rRJQFHFfLVD_hJu21QQAR1njPpg@mail.gmail.com","threadId":"57047","inReplyTo":"xmqqy24a4oyq.fsf@gitster.g","subject":"Re: [PATCH v5 16/16] reftable: be more paranoid about 0-length memcpy calls","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2022-01-12T11:39:49Z","receivedAt":"2022-01-12T11:40:02Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Fri, Dec 24, 2021 at 5:16 AM Junio C Hamano wrote:\n>\n> But your safe_memcpy() should not be\n>\n> safe_memcpy(dst, src, n)\n> {\n> if (n)\n> memcpy(dst, src, n);\n> return dst;\n> }\n>\n> Using memcpy() with size==0 is not a crime wrt the language.\n> Passing an invalid pointer while doing so is.\n\nIt's not a crime, but what is the benefit of calling memcpy with n == 0 ?\n\n> safe_memcpy(dst, src, n)\n> {\n> if (dst)\n> memcpy(dst, src, n);\n> else if (n)\n> BUG(...);\n\nI think this is suboptimal. Sure, a segfault is uglier than \"out of\nmemory\" error, but both effectively crash the program, so the\ndifference isn't that big.\n\nThe nice way is if the reftable library grows an error-code\nREFTABLE_OOM, which is propagated once a malloc or realloc returns\nNULL.\n\nWe could test this exhaustively in the unittest by swapping in a\nmalloc that starts failing after N allocations, and then running a\ntransaction in a loop, increasing N.\nI'll have to look more closely if this is possible throughout, so for\nthis series, I'll just take a closer look at the current call-sites to\nsee if NULL can really occur or not.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"445986","messageId":"CAFQ2z_OFK77TC605GqM2Lw1Lf21fyF2cVKkGVrXO6TP6zcp+mw@mail.gmail.com","threadId":"57047","inReplyTo":"xmqq4k6y63j7.fsf@gitster.g","subject":"Re: [PATCH v5 02/16] reftable: fix resource leak in block.c error path","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2022-01-12T11:58:57Z","receivedAt":"2022-01-12T11:59:10Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Fri, Dec 24, 2021 at 5:16 AM Junio C Hamano wrote:\n> Once you\n> initialize at the declaration with \"less meaningful\" value (like\n> zero initialization), the tools won't be able to tell when the code\n> uses that variable \"uninitialized\" (because the assignment was\n> skipped by a bug), since it appears to always be initialied to them.\n\nWhich tools are these? When I add\n\nstatic void test_memcpy(void)\n{\n uint32_t dest;\n char not_init[200];\n int i;\n memcpy(&dest, not_init, sizeof(dest));\n\n for (i = 0 ; i < 10; i++)\n not_init[i] = rand() % 255 + 1;\n printf(\"%d\", (int) strlen(not_init));\n}\n\nto the C code, it compiles cleanly if I do \"make DEVELOPER=1\".\n\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"446002","messageId":"CAFQ2z_PH5p3KZoHHQ28ouSK3azy_o_wzEObfyQi=z7Bv5o4BRw@mail.gmail.com","threadId":"57047","inReplyTo":"CAFQ2z_PDfreeGttTbNTbnr7rRJQFHFfLVD_hJu21QQAR1njPpg@mail.gmail.com","subject":"Re: [PATCH v5 16/16] reftable: be more paranoid about 0-length memcpy calls","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2022-01-12T12:59:13Z","receivedAt":"2022-01-12T12:59:34Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Wed, Jan 12, 2022 at 12:39 PM Han-Wen Nienhuys wrote:\n> this series, I'll just take a closer look at the current call-sites to\n> see if NULL can really occur or not.\n\nI've dropped this patch from the series. The only suspect callsite is\nin obj_record_decode(),\nand that is better addressed by other changes to make sure we never\ntry to do a 0 length copy.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"446025","messageId":"6993c156-4df1-198f-5ce0-781ba13d5dd2@web.de","threadId":"57047","inReplyTo":"CAFQ2z_OFK77TC605GqM2Lw1Lf21fyF2cVKkGVrXO6TP6zcp+mw@mail.gmail.com","subject":"Re: [PATCH v5 02/16] reftable: fix resource leak in block.c error path","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2022-01-12T14:03:20Z","receivedAt":"2022-01-12T14:03:58Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 12.01.22 um 12:58 schrieb Han-Wen Nienhuys:\n> On Fri, Dec 24, 2021 at 5:16 AM Junio C Hamano wrote:\n>> Once you\n>> initialize at the declaration with \"less meaningful\" value (like\n>> zero initialization), the tools won't be able to tell when the code\n>> uses that variable \"uninitialized\" (because the assignment was\n>> skipped by a bug), since it appears to always be initialied to them.\n>\n> Which tools are these? When I add\n>\n> static void test_memcpy(void)\n> {\n> uint32_t dest;\n> char not_init[200];\n> int i;\n> memcpy(&dest, not_init, sizeof(dest));\n>\n> for (i = 0 ; i < 10; i++)\n> not_init[i] = rand() % 255 + 1;\n> printf(\"%d\", (int) strlen(not_init));\n> }\n>\n> to the C code, it compiles cleanly if I do \"make DEVELOPER=1\".\n\nhttps://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html says about\n-Wuninitialized:\n\n \"Note that there may be no warning about a variable that is used only\n to compute a value that itself is never used, because such\n computations may be deleted by data flow analysis before the\n warnings are printed.\"\n\nAnd indeed, dest is not used above. But even if we change the funtion\nto use dest by returning it, GCC versions 9.1 and higher don't warn\nabout the use of the uninitialized buffer. GCC 4.7.1 to 8.5 do warn,\nthough: https://godbolt.org/z/zYz9KvPdK\n\nRené\n"},{"id":"446114","messageId":"220113.86bl0gvtq3.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"CAFQ2z_OFK77TC605GqM2Lw1Lf21fyF2cVKkGVrXO6TP6zcp+mw@mail.gmail.com","subject":"Re: [PATCH v5 02/16] reftable: fix resource leak in block.c error path","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-13T09:55:54Z","receivedAt":"2022-01-13T10:02:50Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Jan 12 2022, Han-Wen Nienhuys wrote:\n\n> On Fri, Dec 24, 2021 at 5:16 AM Junio C Hamano wrote:\n>> Once you\n>> initialize at the declaration with \"less meaningful\" value (like\n>> zero initialization), the tools won't be able to tell when the code\n>> uses that variable \"uninitialized\" (because the assignment was\n>> skipped by a bug), since it appears to always be initialied to them.\n>\n> Which tools are these? When I add\n>\n> static void test_memcpy(void)\n> {\n> uint32_t dest;\n> char not_init[200];\n> int i;\n> memcpy(&dest, not_init, sizeof(dest));\n>\n> for (i = 0 ; i < 10; i++)\n> not_init[i] = rand() % 255 + 1;\n> printf(\"%d\", (int) strlen(not_init));\n> }\n>\n> to the C code, it compiles cleanly if I do \"make DEVELOPER=1\".\n\nAside from what René said in his follow-up, I think what Junio's\npointing out here has to do with the use of this pattern in general, not\nthe specific code being discussed here.\n\nI.e. if you get into the habit of needless initialization it may not\nmatter right now, but once the function grows an if/else branch, or\nsomeone copies the pattern to such a function it may hide a logic error.\n\nSo it's not about analyzing the control specific flow here, but that\nyour upthread patch is separating a variable and its actual\ninternalization by ~20 lines.\n\nSo in the general case, by initializing it when it's declared it's more\nlikely that we'll introduce a logic error where it won't be initialized\nat all.\n"},{"id":"446154","messageId":"CAFQ2z_NpK3Yr=vHScnJV=1uDnh4P0PceuCXK+CVLvVJ4GtVh3A@mail.gmail.com","threadId":"57047","inReplyTo":"220113.86bl0gvtq3.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v5 02/16] reftable: fix resource leak in block.c error path","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2022-01-13T14:27:32Z","receivedAt":"2022-01-13T14:27:46Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Thu, Jan 13, 2022 at 11:02 AM Ævar Arnfjörð Bjarmason\n wrote:\n>\n> Aside from what René said in his follow-up, I think what Junio's\n> pointing out here has to do with the use of this pattern in general, not\n> the specific code being discussed here.\n>\n> I.e. if you get into the habit of needless initialization it may not\n> matter right now, but once the function grows an if/else branch, or\n> someone copies the pattern to such a function it may hide a logic error.\n>\n> So it's not about analyzing the control specific flow here, but that\n> your upthread patch is separating a variable and its actual\n> internalization by ~20 lines.\n\nI know this is the Git project's preferred style, so I'm OK with\nadapting to that, but I'm also sad about it.\n\nSure, you can introduce logic errors by refactoring things, but with\ninitialized data, you get a reproducible logic error, rather than\nsomething whose failure mode depends on platform, compiler and\nsurrounding function calls. This makes debugging problems across\nplatforms easier. In particular, I don't have a functioning Windows\nenvironment, so anything that helps minimize differences across\nplatforms is welcome to me.\n\n--\nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\nRegistergericht und -nummer: Hamburg, HRB 86891\nSitz der Gesellschaft: Hamburg\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"446174","messageId":"xmqqczkvii3v.fsf@gitster.g","threadId":"57047","inReplyTo":"6993c156-4df1-198f-5ce0-781ba13d5dd2@web.de","subject":"Re: [PATCH v5 02/16] reftable: fix resource leak in block.c error path","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-13T18:52:04Z","receivedAt":"2022-01-13T18:52:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"René Scharfe writes:\n\n> Am 12.01.22 um 12:58 schrieb Han-Wen Nienhuys:\n>> On Fri, Dec 24, 2021 at 5:16 AM Junio C Hamano wrote:\n>>> Once you\n>>> initialize at the declaration with \"less meaningful\" value (like\n>>> zero initialization), the tools won't be able to tell when the code\n>>> uses that variable \"uninitialized\" (because the assignment was\n>>> skipped by a bug), since it appears to always be initialied to them.\n>>\n>> Which tools are these? When I add\n>>\n>> static void test_memcpy(void)\n>> {\n>> uint32_t dest;\n>> char not_init[200];\n>> int i;\n>> memcpy(&dest, not_init, sizeof(dest));\n>>\n>> for (i = 0 ; i < 10; i++)\n>> not_init[i] = rand() % 255 + 1;\n>> printf(\"%d\", (int) strlen(not_init));\n>> }\n>>\n>> to the C code, it compiles cleanly if I do \"make DEVELOPER=1\".\n>\n> https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html says about\n> -Wuninitialized:\n>\n> \"Note that there may be no warning about a variable that is used only\n> to compute a value that itself is never used, because such\n> computations may be deleted by data flow analysis before the\n> warnings are printed.\"\n>\n> And indeed, dest is not used above. But even if we change the funtion\n> to use dest by returning it, GCC versions 9.1 and higher don't warn\n> about the use of the uninitialized buffer. GCC 4.7.1 to 8.5 do warn,\n> though: https://godbolt.org/z/zYz9KvPdK\n\nWhat I meant in the original discussion was a runtime checker that\nnotices a read from an uninitialized location (I've also written one\nmyself way before my Git time, which was how I noticed strcpy() on\nold SunOS, in an attempt to optimize by loading a word at a time,\nsometimes read one word too much beyond the end of a page).\n\nBut if a static analysis may catch itas a potential error, that is\nanother reason to worry about it, too.\n\nThe original code wanted to do\n\n\tchar message[100] = { 0 };\n\n\tfor (i = 0; i < sizeof(message) - 1; i++)\n \tmessage[i] = something();\n\n\tuse_NUL_terminated_string(message);\n\nand it allowed to omit an assigmnent\n\n\tmessage[sizeof(message) - 1] = '\\0';\n\nafter the loop.\n\nAs I already said, I do not care too deeply in a test-only helper\nlike this.\n\nThanks.\n"},{"id":"446195","messageId":"YeDUA6NhHuCUkqXf@camp.crustytoothpaste.net","threadId":"57047","inReplyTo":"220113.86tue7vr6d.gmgdl@evledraar.gmail.com","subject":"Re: v2.35.0 DEVELOPER=1 regression (was: [PATCH] config.mak.dev: specify -std=gnu99 for gcc/clang)","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2022-01-14T01:38:11Z","receivedAt":"2022-01-14T01:38:16Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2022-01-13 at 10:45:49, Ævar Arnfjörð Bjarmason wrote:\n> [I'm aware that Jeff probably won't read this in his sabbatical, just\n> replying to the relevant thread]\n> \n> Something neither of us considered at the time, but which is pretty\n> obvious in retrospect, is that this new -std=gnu99 dosen't only apply to\n> our own code, but any system and library includes that we need.\n> \n> Thus e.g. FreeBSD 13.0 which previously was happy to compile under\n> DEVELOPER=1 will now die because its core libraries use C11-specific\n> code:\n> \n> archive.c:337:35: error: '_Generic' is a C11 extension [-Werror,-Wc11-extensions]\n> strbuf_addstr(&path_in_archive, basename(path));\n> ^\n> /usr/include/libgen.h:61:21: note: expanded from macro 'basename'\n> #define basename(x) __generic(x, const char *, __old_basename, basename)(x)\n> ^\n> /usr/include/sys/cdefs.h:325:2: note: expanded from macro '__generic'\n> _Generic(expr, t: yes, default: no)\n> ^\n\nI think we had this discussion about FreeBSD before and that's why I\nspecifically dropped that option from the main makefile. We can either\ndrop that patch, or we can set it to -std=gnu11 and tell folks setting\nDEVELOPER to use a system released in the last five years. I think we\ncan be a little stricter with what we require in the case of DEVELOPER\nthan we might be otherwise.\n-- \nbrian m. carlson (he/him or they/them)\nToronto, Ontario, CA\n"},{"id":"446213","messageId":"220114.86a6fytte9.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"YeDUA6NhHuCUkqXf@camp.crustytoothpaste.net","subject":"Re: v2.35.0 DEVELOPER=1 regression (was: [PATCH] config.mak.dev: specify -std=gnu99 for gcc/clang)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-14T12:01:06Z","receivedAt":"2022-01-14T12:05:13Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Jan 14 2022, brian m. carlson wrote:\n\n> [[PGP Signed Part:Undecided]]\n> On 2022-01-13 at 10:45:49, Ævar Arnfjörð Bjarmason wrote:\n>> [I'm aware that Jeff probably won't read this in his sabbatical, just\n>> replying to the relevant thread]\n>> \n>> Something neither of us considered at the time, but which is pretty\n>> obvious in retrospect, is that this new -std=gnu99 dosen't only apply to\n>> our own code, but any system and library includes that we need.\n>> \n>> Thus e.g. FreeBSD 13.0 which previously was happy to compile under\n>> DEVELOPER=1 will now die because its core libraries use C11-specific\n>> code:\n>> \n>> archive.c:337:35: error: '_Generic' is a C11 extension [-Werror,-Wc11-extensions]\n>> strbuf_addstr(&path_in_archive, basename(path));\n>> ^\n>> /usr/include/libgen.h:61:21: note: expanded from macro 'basename'\n>> #define basename(x) __generic(x, const char *, __old_basename, basename)(x)\n>> ^\n>> /usr/include/sys/cdefs.h:325:2: note: expanded from macro '__generic'\n>> _Generic(expr, t: yes, default: no)\n>> ^\n>\n> I think we had this discussion about FreeBSD before and that's why I\n> specifically dropped that option from the main makefile. We can either\n> drop that patch, or we can set it to -std=gnu11 and tell folks setting\n> DEVELOPER to use a system released in the last five years. I think we\n> can be a little stricter with what we require in the case of DEVELOPER\n> than we might be otherwise.\n\nI guess, yeah.\n\nAs a practical matter the changes in this cycle have made DEVELOPER=1\nmuch more fragile as a cross-platform facility. I test on various\nplatforms/OS's (across the GCC farm), and before this cycle only AIX and\nSolaris were something I had to pay special attention to.\n\nNow we've got things like this breaking (by default) non-obscure things\nlike FreeBSD.\n\nOf course I can manually set -Wall etc. which is what I previously got\nout of this, but before e.g. pedantic was opt-in, now an effective\nC-version-a-pedantic is the default, and doesn't have a tweaking knob at\nall.\n\nAnyway, we'll see how much of a hassle it is, and it's not too painful\nfor me right now, so I don't think anything needs to be done in the RC\nperiod.\n\nBut I wonder if the X-Y problem we're trying to solve is making sure we\ndon't move past C99 unintentionally whether this wouldn't be better\nsolved by dropping this -std=gnu99 approach in the Makefile, and instead\njust do that in one of the CI jobs (whose OS includes would be known to\nbe C99-OK).\n\n"},{"id":"446244","messageId":"xmqqilumayfh.fsf@gitster.g","threadId":"57047","inReplyTo":"YeDUA6NhHuCUkqXf@camp.crustytoothpaste.net","subject":"Re: v2.35.0 DEVELOPER=1 regression","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-14T19:51:14Z","receivedAt":"2022-01-14T19:51:18Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n>> DEVELOPER=1 will now die because its core libraries use C11-specific\n>> code:\n>> \n>> archive.c:337:35: error: '_Generic' is a C11 extension [-Werror,-Wc11-extensions]\n>> strbuf_addstr(&path_in_archive, basename(path));\n>> ^\n>> /usr/include/libgen.h:61:21: note: expanded from macro 'basename'\n>> #define basename(x) __generic(x, const char *, __old_basename, basename)(x)\n>> ^\n>> /usr/include/sys/cdefs.h:325:2: note: expanded from macro '__generic'\n>> _Generic(expr, t: yes, default: no)\n>> ^\n\nWow, that sounds horribly broken.\n\n> I think we had this discussion about FreeBSD before and that's why I\n> specifically dropped that option from the main makefile. We can either\n> drop that patch, or we can set it to -std=gnu11 and tell folks setting\n> DEVELOPER to use a system released in the last five years. I think we\n> can be a little stricter with what we require in the case of DEVELOPER\n> than we might be otherwise.\n\nBut that is not being stricter, but looser, no? I thought that the\npoint of -std=gnu99 was to allow us to use C99 features while catching\nuse of language features newer than that, and use of -std=gnu11 will\ndefeat half the point, wouldn't it?\n"},{"id":"446249","messageId":"220114.8635lqrqci.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"xmqqilumayfh.fsf@gitster.g","subject":"Re: v2.35.0 DEVELOPER=1 regression","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-14T20:41:07Z","receivedAt":"2022-01-14T20:53:54Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Jan 14 2022, Junio C Hamano wrote:\n\n> \"brian m. carlson\" writes:\n>\n>>> DEVELOPER=1 will now die because its core libraries use C11-specific\n>>> code:\n>>> \n>>> archive.c:337:35: error: '_Generic' is a C11 extension [-Werror,-Wc11-extensions]\n>>> strbuf_addstr(&path_in_archive, basename(path));\n>>> ^\n>>> /usr/include/libgen.h:61:21: note: expanded from macro 'basename'\n>>> #define basename(x) __generic(x, const char *, __old_basename, basename)(x)\n>>> ^\n>>> /usr/include/sys/cdefs.h:325:2: note: expanded from macro '__generic'\n>>> _Generic(expr, t: yes, default: no)\n>>> ^\n>\n> Wow, that sounds horribly broken.\n\nYes, but it's also working as designed :) We're erroring because the C\nlibrary headers on the OS aren't C99-compliant. That it would apply to\nonly git.git's sources was only ever wishful thinking.\n\n>> I think we had this discussion about FreeBSD before and that's why I\n>> specifically dropped that option from the main makefile. We can either\n>> drop that patch, or we can set it to -std=gnu11 and tell folks setting\n>> DEVELOPER to use a system released in the last five years. I think we\n>> can be a little stricter with what we require in the case of DEVELOPER\n>> than we might be otherwise.\n>\n> But that is not being stricter, but looser, no? I thought that the\n> point of -std=gnu99 was to allow us to use C99 features while catching\n> use of language features newer than that, and use of -std=gnu11 will\n> defeat half the point, wouldn't it?\n\nIndeed. But also as noted in my side-thread reply in\n<220114.86a6fytte9.gmgdl@evledraar.gmail.com> I think that being able to\nuse it like this was always wishful thinking.\n\nI.e. to brian's \"tell folks setting DEVELOPER to use a system released\nin the last five years\". This is the exact opposite of that.\n\nWe are implicitly expecting that the OS will forever be using the\n20+-year old C99 standard, and not trying out fetures in C11\nstandardized 10+ years ago. The FreeBSD 13.0 release is less than a year\nold[1].\n\nWhen I went looking I found that the change to use __generic in its\nlibgen.h just so happens to have been made in FreeBSD sources around 5\nyears ago: [2].\n\n1. https://www.freebsd.org/releases/13.0R/announce/\n2. https://github.com/freebsd/freebsd-src/commit/34168b28e99b0bb328c7bb49cd91cb942181056f\n"},{"id":"446257","messageId":"xmqqk0f29e6w.fsf@gitster.g","threadId":"57047","inReplyTo":"220114.8635lqrqci.gmgdl@evledraar.gmail.com","subject":"Re: v2.35.0 DEVELOPER=1 regression","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-14T21:53:43Z","receivedAt":"2022-01-14T21:53:48Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n>> Wow, that sounds horribly broken.\n>\n> Yes, but it's also working as designed :) We're erroring because the C\n> library headers on the OS aren't C99-compliant. That it would apply to\n> only git.git's sources was only ever wishful thinking.\n\nNo, C library supporting only C11 is perfectly fine. On such a\nsystem, the compiler shouldn't even support -std=gnu99. That is\nwhat I consider broken.\n"},{"id":"446262","messageId":"xmqqzgny7xo7.fsf@gitster.g","threadId":"57047","inReplyTo":"xmqqilumayfh.fsf@gitster.g","subject":"Re: v2.35.0 DEVELOPER=1 regression","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-14T22:35:52Z","receivedAt":"2022-01-14T22:35:58Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n>> I think we had this discussion about FreeBSD before and that's why I\n>> specifically dropped that option from the main makefile. We can either\n>> drop that patch, or we can set it to -std=gnu11 and tell folks setting\n>> DEVELOPER to use a system released in the last five years. I think we\n>> can be a little stricter with what we require in the case of DEVELOPER\n>> than we might be otherwise.\n>\n> But that is not being stricter, but looser, no? I thought that the\n> point of -std=gnu99 was to allow us to use C99 features while catching\n> use of language features newer than that, and use of -std=gnu11 will\n> defeat half the point, wouldn't it?\n\nIf FreeBSD (or any other platform) cannot do \"reject features beyond\nC99\", I am perfectly OK to drop -std=gnu99 on such a place.\n\nDEVELOPER=YesPlease ought to be a collection of settings that helps\nthe developers the most. So on platforms that *can* do \"reject\nfeatures beyond C99\", I prefer to have it enabled when\nDEVELOPER=YesPlease is given.\n\nIt seems that -std=gnu99 is only added conditionally even in today's\nconfig.mak.dev, so it is fine if we dropped -std=gnu99 from there.\nWhich means that developers on FreeBSD cannot participate in vetting\nuse of features beyond C99, but there are developers on other\nplatforms who will, so it's not too bad, I would say.\n\nAs config.mak.dev is included after config.mak.uname, something like\nthis may be sufficient, perhaps?\n\n config.mak.dev | 5 +++++\n 1 file changed, 5 insertions(+)\n\ndiff --git i/config.mak.dev w/config.mak.dev\nindex d4afac6b51..3deb076d5e 100644\n--- i/config.mak.dev\n+++ w/config.mak.dev\n@@ -20,9 +20,14 @@ endif\n endif\n endif\n \n+ifneq ($(uname_S),FreeBSD)\n ifneq ($(or $(filter gcc6,$(COMPILER_FEATURES)),$(filter clang7,$(COMPILER_FEATURES))),)\n DEVELOPER_CFLAGS += -std=gnu99\n endif\n+else\n+# FreeBSD cannot limit to C99 because its system headers unconditionally\n+# rely on C11 features.\n+endif\n \n DEVELOPER_CFLAGS += -Wdeclaration-after-statement\n DEVELOPER_CFLAGS += -Wformat-security\n"},{"id":"446267","messageId":"xmqqh7a598g3.fsf@gitster.g","threadId":"57047","inReplyTo":"xmqqk0f29e6w.fsf@gitster.g","subject":"Re: v2.35.0 DEVELOPER=1 regression","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-14T23:57:48Z","receivedAt":"2022-01-14T23:57:53Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Ævar Arnfjörð Bjarmason writes:\n>\n>>> Wow, that sounds horribly broken.\n>>\n>> Yes, but it's also working as designed :) We're erroring because the C\n>> library headers on the OS aren't C99-compliant. That it would apply to\n>> only git.git's sources was only ever wishful thinking.\n>\n> No, C library supporting only C11 is perfectly fine. On such a\n> system, the compiler shouldn't even support -std=gnu99. That is\n> what I consider broken.\n\nOr, the system headers should be arranged in such a way that\ndepending on __STDC_VERSION__, it should refrain from using features\nof the language that is not supported. So supporting -std=gnu99 in\ntheir compilers may not be a bug---but in that case, their system\nheaders are buggy.\n\nAnyway. I think shows a viable way\nforward.\n"},{"id":"446268","messageId":"220115.86tue5rhlf.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"xmqqzgny7xo7.fsf@gitster.g","subject":"Re: v2.35.0 DEVELOPER=1 regression","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-14T23:56:34Z","receivedAt":"2022-01-15T00:02:56Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Jan 14 2022, Junio C Hamano wrote:\n\n> Junio C Hamano writes:\n>\n>>> I think we had this discussion about FreeBSD before and that's why I\n>>> specifically dropped that option from the main makefile. We can either\n>>> drop that patch, or we can set it to -std=gnu11 and tell folks setting\n>>> DEVELOPER to use a system released in the last five years. I think we\n>>> can be a little stricter with what we require in the case of DEVELOPER\n>>> than we might be otherwise.\n>>\n>> But that is not being stricter, but looser, no? I thought that the\n>> point of -std=gnu99 was to allow us to use C99 features while catching\n>> use of language features newer than that, and use of -std=gnu11 will\n>> defeat half the point, wouldn't it?\n>\n> If FreeBSD (or any other platform) cannot do \"reject features beyond\n> C99\", I am perfectly OK to drop -std=gnu99 on such a place.\n>\n> DEVELOPER=YesPlease ought to be a collection of settings that helps\n> the developers the most. So on platforms that *can* do \"reject\n> features beyond C99\", I prefer to have it enabled when\n> DEVELOPER=YesPlease is given.\n>\n> It seems that -std=gnu99 is only added conditionally even in today's\n> config.mak.dev, so it is fine if we dropped -std=gnu99 from there.\n> Which means that developers on FreeBSD cannot participate in vetting\n> use of features beyond C99, but there are developers on other\n> platforms who will, so it's not too bad, I would say.\n>\n> As config.mak.dev is included after config.mak.uname, something like\n> this may be sufficient, perhaps?\n\nWasn't the initial goal here to check whether we'd accidentally include\nC99? Just checking on GCC/Clang somewhere seems sufficient enough,\nI.e. something like:\n \n diff --git a/ci/lib.sh b/ci/lib.sh\n index 9d28ab50fb4..94d0d4127b9 100755\n --- a/ci/lib.sh\n +++ b/ci/lib.sh\n @@ -209,6 +209,9 @@ linux-leaks)\n export SANITIZE=leak\n export GIT_TEST_PASSING_SANITIZE_LEAK=true\n ;;\n +linux-gcc|linux-clang)\n + MAKEFLAGS=\"$MAKEFLAGS CFLAGS=-std=gnu99\"\n + ;;\n esac\n \n MAKEFLAGS=\"$MAKEFLAGS CC=${CC:-cc}\"\n diff --git a/config.mak.dev b/config.mak.dev\n index d4afac6b51f..216f92493fc 100644\n --- a/config.mak.dev\n +++ b/config.mak.dev\n @@ -20,10 +20,6 @@ endif\n endif\n endif\n \n -ifneq ($(or $(filter gcc6,$(COMPILER_FEATURES)),$(filter clang7,$(COMPILER_FEATURES))),)\n -DEVELOPER_CFLAGS += -std=gnu99\n -endif\n -\n DEVELOPER_CFLAGS += -Wdeclaration-after-statement\n DEVELOPER_CFLAGS += -Wformat-security\n DEVELOPER_CFLAGS += -Wold-style-definition\n \nWe're just starting to get into the whack-a-mole of hardcoding OS's and\ncompiler versions, it's not all versions of FreeBSD, probably not just\nthat OS etc.\n\nI've also wondered why we can't just ship our own super-light version of\nautoconf to run some real probes, and as a result eventually get rid of\nconfigure.ac, ./detect-compiler, the version hardcoding in\nconfig.mak.dev etc.\n\nJust something as simple as extending this:\n \n diff --git a/Makefile b/Makefile\n index 5580859afdb..f197a07c100 100644\n --- a/Makefile\n +++ b/Makefile\n @@ -2557,6 +2557,15 @@ else\n $(OBJECTS): $(LIB_H) $(GENERATED_H)\n endif\n \n +ifdef DEVELOPER\n +probe/std.mak:\n + @mkdir -p $(@D)\n + $(QUIET_GEN)if $(CC) -E -std=gnu99 - /dev/null; then \\\n + echo 'DEVELOPER_CFLAGS += -std=gnu99'; \\\n + fi >$@\n +include probe/std.mak\n +endif\n +\n ifeq ($(GENERATE_COMPILATION_DATABASE),yes)\n all:: compile_commands.json\n compile_commands.json:\n"},{"id":"446272","messageId":"xmqqsftp7sca.fsf@gitster.g","threadId":"57047","inReplyTo":"220115.86tue5rhlf.gmgdl@evledraar.gmail.com","subject":"Re: v2.35.0 DEVELOPER=1 regression","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-15T00:31:01Z","receivedAt":"2022-01-15T00:31:06Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Wasn't the initial goal here to check whether we'd accidentally include\n> C99? Just checking on GCC/Clang somewhere seems sufficient enough,\n\nWhat do you mean by \"accidentally include\"?\n\nThe goal here is that developer build should give developer a set of\noptions that help the most---\"not going beyond C99\" is something we\nwant them to be checking, if able (and we have established that\nusers on FreeBSD are not capable).\n\nDoesn't your \"something like\" limits the check to CI? Developers\nwith compilers that _can_ help ensuring that we do not go beyond C99\nshould be able to do so by their local developer build.\n\n> I.e. something like:\n> \n> diff --git a/ci/lib.sh b/ci/lib.sh\n> index 9d28ab50fb4..94d0d4127b9 100755\n> --- a/ci/lib.sh\n> +++ b/ci/lib.sh\n> @@ -209,6 +209,9 @@ linux-leaks)\n> export SANITIZE=leak\n> export GIT_TEST_PASSING_SANITIZE_LEAK=true\n> ;;\n> +linux-gcc|linux-clang)\n> + MAKEFLAGS=\"$MAKEFLAGS CFLAGS=-std=gnu99\"\n> + ;;\n> esac\n> \n> MAKEFLAGS=\"$MAKEFLAGS CC=${CC:-cc}\"\n> diff --git a/config.mak.dev b/config.mak.dev\n> index d4afac6b51f..216f92493fc 100644\n> --- a/config.mak.dev\n> +++ b/config.mak.dev\n> @@ -20,10 +20,6 @@ endif\n> endif\n> endif\n> \n> -ifneq ($(or $(filter gcc6,$(COMPILER_FEATURES)),$(filter clang7,$(COMPILER_FEATURES))),)\n> -DEVELOPER_CFLAGS += -std=gnu99\n> -endif\n> -\n> DEVELOPER_CFLAGS += -Wdeclaration-after-statement\n> DEVELOPER_CFLAGS += -Wformat-security\n> DEVELOPER_CFLAGS += -Wold-style-definition\n"},{"id":"446276","messageId":"220115.86h7a5rfdu.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"xmqqsftp7sca.fsf@gitster.g","subject":"Re: v2.35.0 DEVELOPER=1 regression","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-15T00:41:48Z","receivedAt":"2022-01-15T00:50:41Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Jan 14 2022, Junio C Hamano wrote:\n\n> Ævar Arnfjörð Bjarmason writes:\n>\n>> Wasn't the initial goal here to check whether we'd accidentally include\n>> C99? Just checking on GCC/Clang somewhere seems sufficient enough,\n>\n> What do you mean by \"accidentally include\"?\n>\n> The goal here is that developer build should give developer a set of\n> options that help the most---\"not going beyond C99\" is something we\n> want them to be checking, if able (and we have established that\n> users on FreeBSD are not capable).\n\nThe reason we ended up with -std=gnu99 is because of this thread where\nHan-Wen accidentally used an unnamed structs/unions:\nhttps://lore.kernel.org/git/xmqqlf0w5bbc.fsf@gitster.g/\n\nHe then pointed out that it would be nice if CI caught this, which would\nhave been sufficient:\nhttps://lore.kernel.org/git/CAFQ2z_NuOy+-pfSoNAYjJhS9jZCYOfoFue10=k=iyPVsPYrB3g@mail.gmail.com/\n\n> Doesn't your \"something like\" limits the check to CI? Developers\n> with compilers that _can_ help ensuring that we do not go beyond C99\n> should be able to do so by their local developer build.\n\nYes, I'm suggesting that what we're trying to catch here is rare enough\nthat that may be sufficient, and which would sidestep the issue of\nwasting time on trying to make this portable everywhere.\n\nThe DEVELOPER=1 feature has recently gone from something I used on a\nwide varity on of platforms and the worst I got was notices about\nunknown -W flags in a couple of them, to something where recently due to\n-fno-common and now more widely with -std=gnu99 I've ended up having to\nmanually blacklist it on a few boxes where it Just Worked.\n\nSo maybe you think we absolutely need to spend the effort to run less\nportable checks like that locally if at all possible, with all the\nconfig.mak.uname etc. work that entails, if so OK.\n\nI'm just suggesting that perhaps a more practical solution is to\nside-step those portability issues by checking this in CI, which I think\nwould be Good Enough on this case.\n"},{"id":"446280","messageId":"xmqqfspp6c22.fsf@gitster.g","threadId":"57047","inReplyTo":"220115.86h7a5rfdu.gmgdl@evledraar.gmail.com","subject":"Re: v2.35.0 DEVELOPER=1 regression","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-15T01:08:05Z","receivedAt":"2022-01-15T01:08:09Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> I'm just suggesting that perhaps a more practical solution is to\n> side-step those portability issues by checking this in CI, which I think\n> would be Good Enough on this case.\n\nUse of GitHub PR + GGG is required to take advantage of that, no? I\ndo not think I want to see a change that makes it _more_ likely that\na patch I pick up from the list will have problems that my local\nintegration work will not catch (because DEVELOPER=YesPlease build\nis castrated at the original contributor side, and also on my box)\nand will only be caught after I push out 'seen'.\n\n\n"},{"id":"446400","messageId":"nycvar.QRO.7.76.6.2201181308540.2121@tvgsbejvaqbjf.bet","threadId":"57047","inReplyTo":"xmqqzgny7xo7.fsf@gitster.g","subject":"Re: v2.35.0 DEVELOPER=1 regression","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2022-01-18T12:32:03Z","receivedAt":"2022-01-18T12:32:50Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Fri, 14 Jan 2022, Junio C Hamano wrote:\n\n> Junio C Hamano writes:\n>\n> >> I think we had this discussion about FreeBSD before and that's why I\n> >> specifically dropped that option from the main makefile. We can either\n> >> drop that patch, or we can set it to -std=gnu11 and tell folks setting\n> >> DEVELOPER to use a system released in the last five years. I think we\n> >> can be a little stricter with what we require in the case of DEVELOPER\n> >> than we might be otherwise.\n> >\n> > But that is not being stricter, but looser, no? I thought that the\n> > point of -std=gnu99 was to allow us to use C99 features while catching\n> > use of language features newer than that, and use of -std=gnu11 will\n> > defeat half the point, wouldn't it?\n>\n> If FreeBSD (or any other platform) cannot do \"reject features beyond\n> C99\", I am perfectly OK to drop -std=gnu99 on such a place.\n>\n> DEVELOPER=YesPlease ought to be a collection of settings that helps\n> the developers the most. So on platforms that *can* do \"reject\n> features beyond C99\", I prefer to have it enabled when\n> DEVELOPER=YesPlease is given.\n>\n> It seems that -std=gnu99 is only added conditionally even in today's\n> config.mak.dev, so it is fine if we dropped -std=gnu99 from there.\n> Which means that developers on FreeBSD cannot participate in vetting\n> use of features beyond C99, but there are developers on other\n> platforms who will, so it's not too bad, I would say.\n\nPlus, we have CI runs that do help us, thanks to setting `DEVELOPER=1`\n(see https://github.com/git/git/blob/v2.35.0-rc1/ci/lib.sh#L154).\n\n> As config.mak.dev is included after config.mak.uname, something like\n> this may be sufficient, perhaps?\n>\n> config.mak.dev | 5 +++++\n> 1 file changed, 5 insertions(+)\n>\n> diff --git i/config.mak.dev w/config.mak.dev\n> index d4afac6b51..3deb076d5e 100644\n> --- i/config.mak.dev\n> +++ w/config.mak.dev\n> @@ -20,9 +20,14 @@ endif\n> endif\n> endif\n>\n> +ifneq ($(uname_S),FreeBSD)\n> ifneq ($(or $(filter gcc6,$(COMPILER_FEATURES)),$(filter clang7,$(COMPILER_FEATURES))),)\n> DEVELOPER_CFLAGS += -std=gnu99\n> endif\n> +else\n> +# FreeBSD cannot limit to C99 because its system headers unconditionally\n> +# rely on C11 features.\n> +endif\n>\n> DEVELOPER_CFLAGS += -Wdeclaration-after-statement\n> DEVELOPER_CFLAGS += -Wformat-security\n>\n\nI applied this patch on top of the current tip of `seen`, opened a PR at\nhttps://github.com/gitgitgadget/git/pull/1116. The corresponding FreeBSD\nrun is here: https://cirrus-ci.com/task/5867558132776960, and it\nsucceeded.\n\nIn addition, I concur that this is the best we can do, and I really would\nlike to have that patch as soon as possible because it confuses new\ncontributors when their PR builds fail out of no mistake of their own.\n\nPlease count this as a vote for including this patch in -rc2.\n\nThank you,\nDscho\n"},{"id":"446410","messageId":"patch-1.1-06cc12be94d-20220118T151234Z-avarab@gmail.com","threadId":"57047","inReplyTo":"220113.86tue7vr6d.gmgdl@evledraar.gmail.com","subject":"[PATCH] config.mak.dev: fix DEVELOPER=1 on FreeBSD with -std=gnu99","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-18T15:14:17Z","receivedAt":"2022-01-18T15:14:24Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Since the addition of -std=gnu99 to CFLAGS under DEVELOPER=1 in\n5f46385309b (config.mak.dev: specify -std=gnu99 for gcc/clang,\n2021-12-08) the compilation has been broken on newer FreeBSD\nsystems. E.g. on FreeBSD 13 we'll now get this when compiling\narchive.o:\n\n archive.c:337:35: error: '_Generic' is a C11 extension [-Werror,-Wc11-extensions]\n strbuf_addstr(&path_in_archive, basename(path));\n ^\n /usr/include/libgen.h:61:21: note: expanded from macro 'basename'\n #define basename(x) __generic(x, const char *, __old_basename, basename)(x)\n ^\n /usr/include/sys/cdefs.h:325:2: note: expanded from macro '__generic'\n _Generic(expr, t: yes, default: no)\n ^\n\nThis issue is a bug in FreeBSD introduced in\nfreebsd/freebsd-src@62b7f85d4749 (Leave the C11 keywords alone when we\nhave a recent version of GCC., 2014-09-03)[1].\n\nThat commit started conditionally using the C11 _Generic() keyword as\na function of either a __STDC_VERSION__ check, or\n__has_extension(c_generic_selections). The current FreeBSD code in\nsys/cdefs.h being (slightly modified since that commit):\n\n #if (defined(__STDC_VERSION__) && __STDC_VERSION__ >= 201112L) || \\\n __has_extension(c_generic_selections)\n [...]\n\nThe inherent problem is that __has_extension() is a way to ask\nClang (and GCC) whether the compiler supports that feature, but the\n-Wc11-extensions warning will be issued on the basis of the selected\n__STDC_VERSION__. With -std=gnu99 the __has_extension() built-in will\nreturn true, but the warning will still fire.\n\nLet's narrowly work around this by checking whether:\n\n A. We are under FreeBSD\n B. We are using a C standard older than C11\n\nIn that case we'll include sys/cdefs.h, and undefine FreeBSD's\n__generic() wrapper if it's been defined. We'll then load libgen.h,\nand restore whatever __generic definition we temporarily undefined\nearlier.\n\nAn alternate solution would be to simply define NO_LIBGEN_H=Y in\nconfig.mak.uname for FreeBSD, but this way we'll use its OS-provided\nbasename() and dirname(), and in the general case ensure that nothing\nchanges outside of DEVELOPER=1 builds.\n\n1. https://github.com/freebsd/freebsd-src/commit/62b7f85d4749\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n git-compat-util.h | 18 ++++++++++++++++++\n 1 file changed, 18 insertions(+)\n\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex 1229c8296b9..69d9b5f202f 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -318,7 +318,25 @@ static inline int setitimer(int which, const struct itimerval *value, struct iti\n #endif\n \n #ifndef NO_LIBGEN_H\n+/*\n+ * FreeBSD's libgen.h inadvertently requires C11 features, due to its\n+ * sys/cdefs.h using _Generic() if\n+ * __has_extension(c_generic_selections) is true, regardless of\n+ * __STDC_VERSION__....\n+ */\n+#if defined(__FreeBSD__) && __STDC_VERSION__ - 0 < 201112L\n+#include \n+#ifdef __generic\n+#define __fbsd_generic __generic\n+#endif\n+#undef __generic\n+#endif\n #include \n+/* ...continue FreeBSD-specific hack above */\n+#ifdef __fbsd_generic\n+#define __generic __fbsd_generic\n+#undef __fbsd_generic\n+#endif\n #else\n #define basename gitbasename\n char *gitbasename(char *);\n-- \n2.35.0.rc1.862.gadd77378860\n\n"},{"id":"446411","messageId":"220118.86iluhoyio.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"nycvar.QRO.7.76.6.2201181308540.2121@tvgsbejvaqbjf.bet","subject":"Re: v2.35.0 DEVELOPER=1 regression","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-18T15:17:02Z","receivedAt":"2022-01-18T15:27:05Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Jan 18 2022, Johannes Schindelin wrote:\n\n> Hi Junio,\n>\n> On Fri, 14 Jan 2022, Junio C Hamano wrote:\n>\n>> Junio C Hamano writes:\n>>\n>> >> I think we had this discussion about FreeBSD before and that's why I\n>> >> specifically dropped that option from the main makefile. We can either\n>> >> drop that patch, or we can set it to -std=gnu11 and tell folks setting\n>> >> DEVELOPER to use a system released in the last five years. I think we\n>> >> can be a little stricter with what we require in the case of DEVELOPER\n>> >> than we might be otherwise.\n>> >\n>> > But that is not being stricter, but looser, no? I thought that the\n>> > point of -std=gnu99 was to allow us to use C99 features while catching\n>> > use of language features newer than that, and use of -std=gnu11 will\n>> > defeat half the point, wouldn't it?\n>>\n>> If FreeBSD (or any other platform) cannot do \"reject features beyond\n>> C99\", I am perfectly OK to drop -std=gnu99 on such a place.\n>>\n>> DEVELOPER=YesPlease ought to be a collection of settings that helps\n>> the developers the most. So on platforms that *can* do \"reject\n>> features beyond C99\", I prefer to have it enabled when\n>> DEVELOPER=YesPlease is given.\n>>\n>> It seems that -std=gnu99 is only added conditionally even in today's\n>> config.mak.dev, so it is fine if we dropped -std=gnu99 from there.\n>> Which means that developers on FreeBSD cannot participate in vetting\n>> use of features beyond C99, but there are developers on other\n>> platforms who will, so it's not too bad, I would say.\n>\n> Plus, we have CI runs that do help us, thanks to setting `DEVELOPER=1`\n> (see https://github.com/git/git/blob/v2.35.0-rc1/ci/lib.sh#L154).\n>\n>> As config.mak.dev is included after config.mak.uname, something like\n>> this may be sufficient, perhaps?\n>>\n>> config.mak.dev | 5 +++++\n>> 1 file changed, 5 insertions(+)\n>>\n>> diff --git i/config.mak.dev w/config.mak.dev\n>> index d4afac6b51..3deb076d5e 100644\n>> --- i/config.mak.dev\n>> +++ w/config.mak.dev\n>> @@ -20,9 +20,14 @@ endif\n>> endif\n>> endif\n>>\n>> +ifneq ($(uname_S),FreeBSD)\n>> ifneq ($(or $(filter gcc6,$(COMPILER_FEATURES)),$(filter clang7,$(COMPILER_FEATURES))),)\n>> DEVELOPER_CFLAGS += -std=gnu99\n>> endif\n>> +else\n>> +# FreeBSD cannot limit to C99 because its system headers unconditionally\n>> +# rely on C11 features.\n>> +endif\n>>\n>> DEVELOPER_CFLAGS += -Wdeclaration-after-statement\n>> DEVELOPER_CFLAGS += -Wformat-security\n>>\n>\n> I applied this patch on top of the current tip of `seen`, opened a PR at\n> https://github.com/gitgitgadget/git/pull/1116. The corresponding FreeBSD\n> run is here: https://cirrus-ci.com/task/5867558132776960, and it\n> succeeded.\n>\n> In addition, I concur that this is the best we can do, and I really would\n> like to have that patch as soon as possible because it confuses new\n> contributors when their PR builds fail out of no mistake of their own.\n\nI just submitted a more narrow fix in the side-thread[1]. I've tested it\non the FreeBSD 13.0 box I have access to, but perhaps you'd like to test\nit too?\n\n> Please count this as a vote for including this patch in -rc2.\n\nI'd like to have it too, but for context needing to add NO_UNCOMPRESS2=Y\n(which Junio's punted on for the final[2]) is a much more widespread\nissue of needing new post-install build tweaking than this issue that\nonly affects developer builds on FreeBSD.\n\nWhat makes that FreeBSD run a part of the GGG CI? Another and more\nnarrow workaround for that in particular is to have whatever's driving\nit add -Wno-c11-extensions to the CFLAGS.\n\n1. https://lore.kernel.org/git/patch-1.1-06cc12be94d-20220118T151234Z-avarab@gmail.com/\n2. https://lore.kernel.org/git/patch-1.1-9cea01a1395-20220117T170457Z-avarab@gmail.com/\n"},{"id":"446431","messageId":"xmqqbl09t10v.fsf@gitster.g","threadId":"57047","inReplyTo":"patch-1.1-06cc12be94d-20220118T151234Z-avarab@gmail.com","subject":"Re: [PATCH] config.mak.dev: fix DEVELOPER=1 on FreeBSD with -std=gnu99","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-18T17:19:12Z","receivedAt":"2022-01-18T17:19:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> The inherent problem is that __has_extension() is a way to ask\n> Clang (and GCC) whether the compiler supports that feature, but the\n> -Wc11-extensions warning will be issued on the basis of the selected\n> __STDC_VERSION__. With -std=gnu99 the __has_extension() built-in will\n> return true, but the warning will still fire.\n>\n> Let's narrowly work around this by checking whether:\n\nThe end result, if and when a change along this line proves OK for\nall versions of FreeBSD we care about, may be narrow, but as a band\naid to discuss just before the final release, I do not know if\nanybody can sensibly assess the ramifications of such a change,\nexcept for somebody whose primary development environment has been\nFreeBSD for the past 6 months or more.\n\nThis looks way more than we can take comfortably at this point in\nthe release cycle, compared to \"fixing\" the developer build by\nkicking developers on FreeBSD out of the \"we are C99 or below at\nthis point\" enforcement mechanism.\n\n> A. We are under FreeBSD\n> B. We are using a C standard older than C11\n>\n> In that case we'll include sys/cdefs.h, and undefine FreeBSD's\n> __generic() wrapper if it's been defined. We'll then load libgen.h,\n> and restore whatever __generic definition we temporarily undefined\n> earlier.\n>\n> An alternate solution would be to simply define NO_LIBGEN_H=Y in\n> config.mak.uname for FreeBSD, but this way we'll use its OS-provided\n> basename() and dirname(), and in the general case ensure that nothing\n> changes outside of DEVELOPER=1 builds.\n>\n> 1. https://github.com/freebsd/freebsd-src/commit/62b7f85d4749\n>\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> git-compat-util.h | 18 ++++++++++++++++++\n> 1 file changed, 18 insertions(+)\n>\n> diff --git a/git-compat-util.h b/git-compat-util.h\n> index 1229c8296b9..69d9b5f202f 100644\n> --- a/git-compat-util.h\n> +++ b/git-compat-util.h\n> @@ -318,7 +318,25 @@ static inline int setitimer(int which, const struct itimerval *value, struct iti\n> #endif\n> \n> #ifndef NO_LIBGEN_H\n> +/*\n> + * FreeBSD's libgen.h inadvertently requires C11 features, due to its\n> + * sys/cdefs.h using _Generic() if\n> + * __has_extension(c_generic_selections) is true, regardless of\n> + * __STDC_VERSION__....\n> + */\n> +#if defined(__FreeBSD__) && __STDC_VERSION__ - 0 < 201112L\n> +#include \n> +#ifdef __generic\n> +#define __fbsd_generic __generic\n> +#endif\n> +#undef __generic\n> +#endif\n> #include \n> +/* ...continue FreeBSD-specific hack above */\n> +#ifdef __fbsd_generic\n> +#define __generic __fbsd_generic\n> +#undef __fbsd_generic\n> +#endif\n> #else\n> #define basename gitbasename\n> char *gitbasename(char *);\n"},{"id":"446433","messageId":"xmqq1r15szpg.fsf_-_@gitster.g","threadId":"57047","inReplyTo":"nycvar.QRO.7.76.6.2201181308540.2121@tvgsbejvaqbjf.bet","subject":"[PATCH] Makefile: FreeBSD cannot do C99-or-below build","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-18T17:47:39Z","receivedAt":"2022-01-18T17:47:45Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"In \"make DEVELOPER=YesPlease\" builds, we try to help developers to\ncatch as many potential issues as they can by using -Wall and\nturning compilation warnings into errors. In the same spirit, we\nrecently started adding -std=gnu99 to their CFLAGS, so that they can\nnotice when they accidentally used language features beyond C99.\n\nIt however turns out that FreeBSD 13.0 mistakenly uses C11 extension\nin its system header files regardless of what __STDC_VERSION__ says,\nwhich means that the platform (unless we tweak their system headers)\ncannot be used for this purpose.\n\nIt seems that -std=gnu99 is only added conditionally even in today's\nconfig.mak.dev, so it is fine if we dropped -std=gnu99 from there.\nWhich means that developers on FreeBSD cannot participate in vetting\nuse of features beyond C99, but there are developers on other\nplatforms who will, so it's not too bad.\n\nWe might want a more \"fundamental\" fix to make the platform capable\nof taking -std=gnu99, like working around the use of unconditional\nC11 extension in its system header files by supplying a set of\n\"replacement\" definitions in our header files. We chose not to\npursue such an approach for two reasons at this point:\n\n (1) The fix belongs to the FreeBSD project, not this project, and\n such an upstream fix may happen hopefully in a not-too-distant\n future.\n\n (2) Fixing such a bug in system header files and working it around\n can lead to unexpected breakages (other parts of their system\n header files may not be expecting to see and do not work well\n with our \"replacement\" definitions). This close to the final\n release of this cycle, we have no time for that.\n\nSigned-off-by: Junio C Hamano \n---\n config.mak.dev | 5 +++++\n 1 file changed, 5 insertions(+)\n\ndiff --git a/config.mak.dev b/config.mak.dev\nindex d4afac6b51..3deb076d5e 100644\n--- a/config.mak.dev\n+++ b/config.mak.dev\n@@ -20,9 +20,14 @@ endif\n endif\n endif\n \n+ifneq ($(uname_S),FreeBSD)\n ifneq ($(or $(filter gcc6,$(COMPILER_FEATURES)),$(filter clang7,$(COMPILER_FEATURES))),)\n DEVELOPER_CFLAGS += -std=gnu99\n endif\n+else\n+# FreeBSD cannot limit to C99 because its system headers unconditionally\n+# rely on C11 features.\n+endif\n \n DEVELOPER_CFLAGS += -Wdeclaration-after-statement\n DEVELOPER_CFLAGS += -Wformat-security\n-- \n2.35.0-rc1-135-g45b839adb0\n\n"},{"id":"446446","messageId":"xmqqh7a0ssuh.fsf@gitster.g","threadId":"57047","inReplyTo":"220118.86iluhoyio.gmgdl@evledraar.gmail.com","subject":"Re: v2.35.0 DEVELOPER=1 regression","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-18T20:15:50Z","receivedAt":"2022-01-18T20:15:55Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> I'd like to have it too, but for context needing to add NO_UNCOMPRESS2=Y\n> (which Junio's punted on for the final[2]) is a much more widespread\n> issue of needing new post-install build tweaking than this issue that\n> only affects developer builds on FreeBSD.\n\nI hate it when people misrepresent what I said, even in an attempt\nto spite me. For the \"check ZLIB_VERSION_NUM\" topic, I gave you\nspecific things that needs to be different from the version posted\nwith reasons, fully expecting that a better version of the patch to\nmaterialize soon (knowing how proliferate you can be when you want\nto) to give us enough time to assess the potential damage.\n\nI wouldn't call that \"punted\".\n\nFor this one, config.mak.dev patch WOULD only affect developer\nbuilds, which is a much better solution than overriding what their\nsystem headers want to do and force compiling with C99 mode. With\nthe status quo with today's code, with or without the patch Dscho\nwants in this thread, means ANYBODY will be stopped when they\nattempt to build with -std=gnu99 on FreeBSD, which is a GOOD thing.\n\nThe reason why it is a much better solution to PUNT on using C99\nmode on FreeBSD is because this episode makes it very clear that\nnobody tested building anything that use basename(), dirname(),\netc. with C99 mode on the platform. I do not trust such a build,\neven if we could work around the system header breakage.\n\nThis time we got lucky and wereq stopped by a compilation error, but\nI have a strong suspicion that C99-only mode of compiler on this\nplatform is not as well battle tested as their standard mode of\ncompilation that allows C11. I simply do not think we want to waste\ndeveloper's time with C99-only mode on this platform which may end\nup debugging the platform and not our program. Next bug that will\nhit us may not be so friendly to clearly break compilation.\nInstead, the symptom may be a subtle runtime breakage that wastes\npeople's time.\n\nAfter developers who work on FreeBSD (not Git developers who uses\nFreeBSD) ships an updated system headers so that more programs, not\njust us, are built and testsd with C99-only mode, perhaps it becomes\nusable as a platform to catch use of language features beyond C99,\nlike everybody else. But with such a clearly broken system headers,\nI do not think today's FreeBSD is ready for that. I do trust their\ndefault settings that allows C11, a lot more than C99-only\ncompilation with the \"their libgen is broken, so here is a user side\nworkaround\" patch.\n\n"},{"id":"446455","messageId":"20220118214720.GA8652@neerajsi-x1.localdomain","threadId":"57047","inReplyTo":"xmqq1r15szpg.fsf_-_@gitster.g","subject":"Re: [PATCH] Makefile: FreeBSD cannot do C99-or-below build","fromName":"Neeraj Singh","fromEmail":"nksingh85@gmail.com","sentAt":"2022-01-18T21:47:20Z","receivedAt":"2022-01-18T21:47:25Z","isPatch":true,"sender":{"key":"nksingh85@gmail.com","avatar":null},"body":"The approach of building with C11 on FreeBSD is a good one\ncompared to trying to hack around the headers.\n\nIt appears more like a compiler bug that's being worked around\nhere. The FreeBSD header supposedly uses a GCC extension if the\nC standard version is less than C11. See: \nhttps://github.com/freebsd/freebsd-src/blob/1e7b5f950b2d54ddb257d008592563c4d753aa54/sys/sys/cdefs.h#L317\n\nSigned-off-by: Neeraj Singh \n"},{"id":"446469","messageId":"220119.861r14pomb.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"20220118214720.GA8652@neerajsi-x1.localdomain","subject":"Re: [PATCH] Makefile: FreeBSD cannot do C99-or-below build","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-18T23:36:21Z","receivedAt":"2022-01-19T00:15:31Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Jan 18 2022, Neeraj Singh wrote:\n\n> The approach of building with C11 on FreeBSD is a good one\n> compared to trying to hack around the headers.\n\nThe \"hack around the headers\" suggests that you've seen my\nalternate[1]....\n\n> It appears more like a compiler bug that's being worked around\n> here. The FreeBSD header supposedly uses a GCC extension if the\n> C standard version is less than C11. See: \n> https://github.com/freebsd/freebsd-src/blob/1e7b5f950b2d54ddb257d008592563c4d753aa54/sys/sys/cdefs.h#L317\n\n...which discusses how the line you're linking to here and\n__has_extension() interact, it's not a compiler bug, unless I'm missing\nsomething.\n\nI.e. it's just a FreeBSD include asking the compiler the wrong question\nand/or not suppressing the warning (e.g. via \"#pragma clang\nsystem_header\").\n\n> Signed-off-by: Neeraj Singh \n\nAccidentally addition?\n\n1. https://lore.kernel.org/git/patch-1.1-06cc12be94d-20220118T151234Z-avarab@gmail.com/\n"},{"id":"446470","messageId":"xmqqzgnspob6.fsf@gitster.g","threadId":"57047","inReplyTo":"220119.861r14pomb.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH] Makefile: FreeBSD cannot do C99-or-below build","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-19T00:22:05Z","receivedAt":"2022-01-19T00:22:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> On Tue, Jan 18 2022, Neeraj Singh wrote:\n>\n>> The approach of building with C11 on FreeBSD is a good one\n>> compared to trying to hack around the headers.\n>\n> The \"hack around the headers\" suggests that you've seen my\n> alternate[1]....\n>\n>> It appears more like a compiler bug that's being worked around\n>> here. The FreeBSD header supposedly uses a GCC extension if the\n>> C standard version is less than C11. See: \n>> https://github.com/freebsd/freebsd-src/blob/1e7b5f950b2d54ddb257d008592563c4d753aa54/sys/sys/cdefs.h#L317\n>\n> ...which discusses how the line you're linking to here and\n> __has_extension() interact, it's not a compiler bug, unless I'm missing\n> something.\n>\n> I.e. it's just a FreeBSD include asking the compiler the wrong question\n> and/or not suppressing the warning (e.g. via \"#pragma clang\n> system_header\").\n\nYup, I do agree with you that this looks more like a bug in the\nheader files.\n"},{"id":"446473","messageId":"220119.86sftko9fu.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"xmqqbl09t10v.fsf@gitster.g","subject":"Re: [PATCH] config.mak.dev: fix DEVELOPER=1 on FreeBSD with -std=gnu99","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-19T00:26:19Z","receivedAt":"2022-01-19T00:28:43Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Jan 18 2022, Junio C Hamano wrote:\n\n> Ævar Arnfjörð Bjarmason writes:\n>\n>> The inherent problem is that __has_extension() is a way to ask\n>> Clang (and GCC) whether the compiler supports that feature, but the\n>> -Wc11-extensions warning will be issued on the basis of the selected\n>> __STDC_VERSION__. With -std=gnu99 the __has_extension() built-in will\n>> return true, but the warning will still fire.\n>>\n>> Let's narrowly work around this by checking whether:\n>\n> The end result, if and when a change along this line proves OK for\n> all versions of FreeBSD we care about, may be narrow, but as a band\n> aid to discuss just before the final release, I do not know if\n> anybody can sensibly assess the ramifications of such a change,\n> except for somebody whose primary development environment has been\n> FreeBSD for the past 6 months or more.\n\nlibgen.h is a trivially sized header (~20 lines of non-comment\nnon-whitespace), and uses __generic for exactly one thing.\n\nIf you'd like to go with your patch instead that's fair enough, but I\ndon't see what using FreeBSD as a primary development environment for an\nextended time would buy anyone in understanding this narrow issue\ncausing the C11 warning.\n\n> This looks way more than we can take comfortably at this point in\n> the release cycle, compared to \"fixing\" the developer build by\n> kicking developers on FreeBSD out of the \"we are C99 or below at\n> this point\" enforcement mechanism.\n>\n>> A. We are under FreeBSD\n>> B. We are using a C standard older than C11\n>>\n>> In that case we'll include sys/cdefs.h, and undefine FreeBSD's\n>> __generic() wrapper if it's been defined. We'll then load libgen.h,\n>> and restore whatever __generic definition we temporarily undefined\n>> earlier.\n>>\n>> An alternate solution would be to simply define NO_LIBGEN_H=Y in\n>> config.mak.uname for FreeBSD, but this way we'll use its OS-provided\n>> basename() and dirname(), and in the general case ensure that nothing\n>> changes outside of DEVELOPER=1 builds.\n>>\n>> 1. https://github.com/freebsd/freebsd-src/commit/62b7f85d4749\n>>\n>> Signed-off-by: Ævar Arnfjörð Bjarmason \n>> ---\n>> git-compat-util.h | 18 ++++++++++++++++++\n>> 1 file changed, 18 insertions(+)\n>>\n>> diff --git a/git-compat-util.h b/git-compat-util.h\n>> index 1229c8296b9..69d9b5f202f 100644\n>> --- a/git-compat-util.h\n>> +++ b/git-compat-util.h\n>> @@ -318,7 +318,25 @@ static inline int setitimer(int which, const struct itimerval *value, struct iti\n>> #endif\n>> \n>> #ifndef NO_LIBGEN_H\n>> +/*\n>> + * FreeBSD's libgen.h inadvertently requires C11 features, due to its\n>> + * sys/cdefs.h using _Generic() if\n>> + * __has_extension(c_generic_selections) is true, regardless of\n>> + * __STDC_VERSION__....\n>> + */\n>> +#if defined(__FreeBSD__) && __STDC_VERSION__ - 0 < 201112L\n>> +#include \n>> +#ifdef __generic\n>> +#define __fbsd_generic __generic\n>> +#endif\n>> +#undef __generic\n>> +#endif\n>> #include \n>> +/* ...continue FreeBSD-specific hack above */\n>> +#ifdef __fbsd_generic\n>> +#define __generic __fbsd_generic\n>> +#undef __fbsd_generic\n>> +#endif\n>> #else\n>> #define basename gitbasename\n>> char *gitbasename(char *);\n\n"},{"id":"446476","messageId":"220119.86o848o8ao.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"xmqqh7a0ssuh.fsf@gitster.g","subject":"Re: v2.35.0 DEVELOPER=1 regression","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-19T00:29:47Z","receivedAt":"2022-01-19T00:53:24Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Jan 18 2022, Junio C Hamano wrote:\n\n> Ævar Arnfjörð Bjarmason writes:\n>\n>> I'd like to have it too, but for context needing to add NO_UNCOMPRESS2=Y\n>> (which Junio's punted on for the final[2]) is a much more widespread\n>> issue of needing new post-install build tweaking than this issue that\n>> only affects developer builds on FreeBSD.\n>\n> I hate it when people misrepresent what I said, even in an attempt\n> to spite me.\n\nLet's assume some mutual good faith here. I'm just trying to help debug\nlast-minute issues in this release to help get it out the door.\n\nIf I've misrepresented your views that's clearly true if you're the one\nclaiming it, but it's not intentional.\n\nI was trying to read the tea leaves here in terms of helping patch\ntriage along. My assessment of that NO_UNCOMPRESS2=Y issue from the\nboxes I've tested on is that it's a much more widespread problem new in\nthis release (this is purely from testing on the GCC farm, however\nrepresentative that is). It'll impact multiple OS's, linux distros &\nversions.\n\nWhereas the C11 warning is \"just\" recent FreeBSD && DEVELOPER=1.\n\nSo I assumed if you weren't interested in the former before the final\nyou probably wouldn't be in the latter, but wanted to provide a more\nnarrow fix in case you were.\n\n> For the \"check ZLIB_VERSION_NUM\" topic, I gave you\n> specific things that needs to be different from the version posted\n> with reasons, fully expecting that a better version of the patch to\n> materialize soon (knowing how proliferate you can be when you want\n> to) to give us enough time to assess the potential damage.\n\nI can re-roll it sooner than later if you'd like, but figured per your\n\"I like the basic idea of the patch, but I am afraid that it is way too\nlate in the cycle\"[1] that the resulting on-list distraction wouldn't be\nworth it at this point, if it wasn't being considered for being applied\nin the release window.\n\nAside from \"I can re-roll it\". I also think that the point of making\nthat change mostly (but not entirely) disappears if it isn't included in\nv2.35.0.\n\nI.e. the point of doing it is to avoid the one-time pain of anyone\nbuilding new releases of git on $OLD_OS/$OLD_DISTRO not having to run\ninto the compilation error that's fixed with NO_UNCOMPRESS2=Y.\n\nSo if we put out a release without it anyone who's compiling git\nreleases will need to adjust their build system for that anyway (or\nthey're using autoconf, where it'll Just Work).\n\nIf we then get this into v2.36.0 there'll be someone somewhere that\nbenefits, but I'd think the ship has sailed for most of those who'd\navoid the needless flag twiddling (git-packagers@ et al).\n\nSo I don't know if it's even worth it to pursue that patch in that\ncase...\n\n> I wouldn't call that \"punted\".\n\nI don't think I've used or heard that word outside of this ML. FWIW I'm\nusing it in the sense of \"kicking the can down the road\" or \"deferred\nit\".\n\nMaybe that's incorrect, and I certainly stand corrected if I implied\nsomething that wasn't true about our views by using that word.\n\n> For this one, config.mak.dev patch WOULD only affect developer\n> builds, which is a much better solution than overriding what their\n> system headers want to do and force compiling with C99 mode. With\n> the status quo with today's code, with or without the patch Dscho\n> wants in this thread, means ANYBODY will be stopped when they\n> attempt to build with -std=gnu99 on FreeBSD, which is a GOOD thing.\n>\n> The reason why it is a much better solution to PUNT on using C99\n> mode on FreeBSD is because this episode makes it very clear that\n> nobody tested building anything that use basename(), dirname(),\n> etc. with C99 mode on the platform. I do not trust such a build,\n> even if we could work around the system header breakage.\n\nAside from anything else I think you're assuming a lot about warning\nhygiene in a typical C project. Git's use of even optional -Werror is\nfairly unusual.\n\nIf you Google search that error you'll find numerous past reports of it,\nit just hasn't been solved.\n\nI also think we discussed around the -std=gnu99 change that the \"C99\nmode\" in compilers we tested isn't impacting generated code (although of\ncourse it will if it's conditional on __STDC_VERSION___). Otherwise the\ngenerated objects are the same, it's just what warnings/errors you get\nthat changes.\n\nIn this case the fallback case already existed without my __generic\nhack, so forcing that codepath for libgen.h isn't new.\n\n> This time we got lucky and wereq stopped by a compilation error, but\n> I have a strong suspicion that C99-only mode of compiler on this\n> platform is not as well battle tested as their standard mode of\n> compilation that allows C11. I simply do not think we want to waste\n> developer's time with C99-only mode on this platform which may end\n> up debugging the platform and not our program. Next bug that will\n> hit us may not be so friendly to clearly break compilation.\n> Instead, the symptom may be a subtle runtime breakage that wastes\n> people's time.\n\nI'd share that opinion if this was aCC on HP/UX or xlc on AIX or\nwhatever. But in this case it's just vanilla clang, not some scary\nplatform-specific compiler. Its \"C99-only mode\" isn't anything different\non FreeBSD than Linux.\n\nWe're just talking about one specific and avoidable warning in\n/usr/include/libgen.h on that OS, once we're past that its \"C99-mode\"\nworks just fine.\n\n> After developers who work on FreeBSD (not Git developers who uses\n> FreeBSD) ships an updated system headers so that more programs, not\n> just us, are built and testsd with C99-only mode, perhaps it becomes\n> usable as a platform to catch use of language features beyond C99,\n> like everybody else. But with such a clearly broken system headers,\n> I do not think today's FreeBSD is ready for that. I do trust their\n> default settings that allows C11, a lot more than C99-only\n> compilation with the \"their libgen is broken, so here is a user side\n> workaround\" patch.\n\nFair enough. I'm happy for us to get past this one way or the other,\nwhether it's with your patch or mine.\n\nI took your previous reservations about my suggestions to just do this\nC99 checking in CI-only to mean that you'd like to keep -std=gnu99 on\nDEVELOPER=1 if at all possible, which my approach does.\n\n1. https://lore.kernel.org/git/xmqq7dayw732.fsf@gitster.g/\n"},{"id":"446478","messageId":"xmqq7dawpmfq.fsf@gitster.g","threadId":"57047","inReplyTo":"220119.86o848o8ao.gmgdl@evledraar.gmail.com","subject":"Re: v2.35.0 DEVELOPER=1 regression","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-19T01:02:33Z","receivedAt":"2022-01-19T01:02:37Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Whereas the C11 warning is \"just\" recent FreeBSD && DEVELOPER=1.\n>\n> So I assumed if you weren't interested in the former before the final\n> you probably wouldn't be in the latter, but wanted to provide a more\n> narrow fix in case you were.\n\nIf we muck with the inclusion of libgen.h, it then becomes a problem\nfor everybody who builds on FreeBSD, not just the developer builds.\nIOW, it is not even narrower to begin with. Giving the same\npotential breakage to everybody will make it easier to diagnose it,\nbut because I do not trust -std=gnu99 on today's FreeBSD, I think it\nis a problem we do not even have to solve.\n\n> I.e. the point of doing it is to avoid the one-time pain of anyone\n> building new releases of git on $OLD_OS/$OLD_DISTRO not having to run\n> into the compilation error that's fixed with NO_UNCOMPRESS2=Y.\n> ...\n> If we then get this into v2.36.0 there'll be someone somewhere that\n> benefits, but I'd think the ship has sailed for most of those who'd\n> avoid the needless flag twiddling (git-packagers@ et al).\n\nI actually think it is a good thing. It is what they brought onto\nthemselves. They can follow David's example next time.\n"},{"id":"446480","messageId":"220119.86fspko7fo.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"xmqq7dawpmfq.fsf@gitster.g","subject":"Re: v2.35.0 DEVELOPER=1 regression","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-19T01:05:25Z","receivedAt":"2022-01-19T01:12:00Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Jan 18 2022, Junio C Hamano wrote:\n\n> Ævar Arnfjörð Bjarmason writes:\n>\n>> Whereas the C11 warning is \"just\" recent FreeBSD && DEVELOPER=1.\n>>\n>> So I assumed if you weren't interested in the former before the final\n>> you probably wouldn't be in the latter, but wanted to provide a more\n>> narrow fix in case you were.\n>\n> If we muck with the inclusion of libgen.h, it then becomes a problem\n> for everybody who builds on FreeBSD, not just the developer builds.\n> IOW, it is not even narrower to begin with. Giving the same\n> potential breakage to everybody will make it easier to diagnose it,\n> but because I do not trust -std=gnu99 on today's FreeBSD, I think it\n> is a problem we do not even have to solve.\n\nThe patch I submitted could trivially have an additional \"#ifdef\nDEVELOPER\" or equivalent, which would narrow it down even further.\n\nThe reason it doesn't is because we provide -std=gnu99 in\nconfig.mak.dev, but that doesn't mean that we don't have to deal with\ne.g. a user-supplied -std=gnu99.\n\nExcept of course if we declare that we're not going to generally support\nsuch a thing, and you should only provide such flags via\nDEVELOPER/DEVOPTS, not manually in CFLAGS, which is fair enough.\n\n>> I.e. the point of doing it is to avoid the one-time pain of anyone\n>> building new releases of git on $OLD_OS/$OLD_DISTRO not having to run\n>> into the compilation error that's fixed with NO_UNCOMPRESS2=Y.\n>> ...\n>> If we then get this into v2.36.0 there'll be someone somewhere that\n>> benefits, but I'd think the ship has sailed for most of those who'd\n>> avoid the needless flag twiddling (git-packagers@ et al).\n>\n> I actually think it is a good thing. It is what they brought onto\n> themselves. They can follow David's example next time.\n\nDo you mean David Aguilar's addition of \"NO_UNCOMPRESS2 = YesPlease\" in\n[1]? Isn't that mutually exclusive with pursuing my change to\nauto-detect it[2] instead?\n\nAs noted I'm a bit \"meh\" on my own patch if it's not in the release. I'm\njust trying to see where you stand, since you seemed to want a re-roll\nof it post-release....\n\n1. https://lore.kernel.org/git/20220116020520.26895-1-davvid@gmail.com/\n2. https://lore.kernel.org/git/patch-1.1-9cea01a1395-20220117T170457Z-avarab@gmail.com/\n"},{"id":"446482","messageId":"xmqqwniwo738.fsf@gitster.g","threadId":"57047","inReplyTo":"220119.86fspko7fo.gmgdl@evledraar.gmail.com","subject":"Re: v2.35.0 DEVELOPER=1 regression","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-19T01:19:23Z","receivedAt":"2022-01-19T01:19:29Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> As noted I'm a bit \"meh\" on my own patch if it's not in the release. I'm\n> just trying to see where you stand, since you seemed to want a re-roll\n> of it post-release....\n\nI was hoping to see a quick reroll that would be in time for the\nrelease, but it seems it was overly optimistic back when I gave my\ninitial review. I do not see it happening given that I'd be tagging\nthe -rc2 tomorrow.\n\nI do think in the longer term, we should aim to retire the current\nmechanism and depend more on ZLIB_VERSION_NUM. I am not optimistic\nthat we can have a reasonable version before the final.\n"},{"id":"446562","messageId":"pull.1152.v6.git.git.1642691534.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v5.git.git.1640199396.gitgitgadget@gmail.com","subject":"[PATCH v6 00/15] Reftable coverity fixes","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-20T15:11:59Z","receivedAt":"2022-01-20T15:12:19Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"This series was targeted to 'master'.\n\nThis addresses some complaints from Coverity that Peff reported.\n\nv6:\n\n * drop 0-length memcpy paranoia patch.\n\nHan-Wen Nienhuys (15):\n reftable: fix OOB stack write in print functions\n reftable: fix resource leak in block.c error path\n reftable: fix resource leak blocksource.c\n reftable: check reftable_stack_auto_compact() return value\n reftable: ignore remove() return value in stack_test.c\n reftable: fix resource warning\n reftable: all xxx_free() functions accept NULL arguments\n reftable: order unittests by complexity\n reftable: drop stray printf in readwrite_test\n reftable: handle null refnames in reftable_ref_record_equal\n reftable: make reftable-record.h function signatures const correct\n reftable: implement record equality generically\n reftable: remove outdated file reftable.c\n reftable: make reftable_record a tagged union\n reftable: add print functions to the record types\n\n reftable/block.c | 28 ++-\n reftable/block_test.c | 22 +--\n reftable/blocksource.c | 6 +-\n reftable/generic.c | 41 +++--\n reftable/iter.c | 4 +-\n reftable/merged.c | 33 ++--\n reftable/pq.c | 3 +-\n reftable/pq_test.c | 27 ++-\n reftable/reader.c | 112 ++++++------\n reftable/readwrite_test.c | 67 ++++++-\n reftable/record.c | 366 ++++++++++++++++++++++++-------------\n reftable/record.h | 49 ++---\n reftable/record_test.c | 197 ++++++++++----------\n reftable/reftable-record.h | 14 +-\n reftable/reftable.c | 115 ------------\n reftable/stack.c | 10 +-\n reftable/stack_test.c | 3 +-\n reftable/writer.c | 43 +++--\n t/helper/test-reftable.c | 9 +-\n 19 files changed, 620 insertions(+), 529 deletions(-)\n delete mode 100644 reftable/reftable.c\n\n\nbase-commit: 50b2d72e110cad39ecaf2322bfdf1b60cd13dd96\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1152%2Fhanwen%2Freftable-coverity-v6\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1152/hanwen/reftable-coverity-v6\nPull-Request: https://github.com/git/git/pull/1152\n\nRange-diff vs v5:\n\n 1: e7f1be7bbec = 1: 9218bd59b25 reftable: fix OOB stack write in print functions\n 2: 9ab631a3b29 ! 2: 315ce62e710 reftable: fix resource leak in block.c error path\n @@ reftable/reader.c: int reader_init_block_reader(struct reftable_reader *r, struc\n +\terr = block_reader_init(br, &block, header_off, r->block_size,\n +\t\t\t\thash_size(r->hash_id));\n +done:\n -+\tif (err)\n -+\t\treftable_block_done(&block);\n ++\treftable_block_done(&block);\n +\n +\treturn err;\n }\n @@ reftable/readwrite_test.c: static void test_log_write_read(void)\n +\t\t},\n +\t};\n +\n -+\tfor (i = 0; i < sizeof(message)-1; i++)\n ++\tfor (i = 0; i < sizeof(message) - 1; i++)\n +\t\tmessage[i] = (uint8_t)(rand() % 64 + ' ');\n +\n +\treftable_writer_set_limits(w, 1, 1);\n 3: 8301000e83b ! 3: 7cf65e141cf reftable: fix resource leak blocksource.c\n @@ Metadata\n ## Commit message ##\n reftable: fix resource leak blocksource.c\n \n - This would be triggered in the unlikely event of fstat() failing on an opened\n - file.\n + This would be triggered in the unlikely event of fstat() failing on an\n + opened file.\n \n Signed-off-by: Han-Wen Nienhuys \n \n 4: e6e53aabc39 = 4: 8740ba92981 reftable: check reftable_stack_auto_compact() return value\n 5: 7551bcdd917 = 5: 08be6d90a48 reftable: ignore remove() return value in stack_test.c\n 6: 700a4e247e7 ! 6: 630f67bdb51 reftable: fix resource warning\n @@ Metadata\n ## Commit message ##\n reftable: fix resource warning\n \n - This would trigger in the unlikely event that we are compacting, and the next\n - available file handle is 0.\n + This would trigger in the unlikely event that we are compacting, and\n + the next available file handle is 0.\n \n Signed-off-by: Han-Wen Nienhuys \n \n @@ reftable/stack.c: static int stack_compact_range(struct reftable_stack *st, int\n \thave_lock = 1;\n \terr = stack_uptodate(st);\n @@ reftable/stack.c: static int stack_compact_range(struct reftable_stack *st, int first, int last,\n - \n - \t\tsublock_file_fd = open(subtab_lock.buf,\n - \t\t\t\t O_EXCL | O_CREAT | O_WRONLY, 0644);\n --\t\tif (sublock_file_fd > 0) {\n -+\t\tif (sublock_file_fd >= 0) {\n - \t\t\tclose(sublock_file_fd);\n - \t\t} else if (sublock_file_fd < 0) {\n - \t\t\tif (errno == EEXIST) {\n -@@ reftable/stack.c: static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \t\tgoto done;\n \t}\n \terr = close(lock_file_fd);\n 7: 30016fee4c8 = 7: afee67fb60d reftable: all xxx_free() functions accept NULL arguments\n 8: f90846cc103 ! 8: 6f4f24c17ad reftable: order unittests by complexity\n @@ Metadata\n ## Commit message ##\n reftable: order unittests by complexity\n \n - This is a more practical ordering when working on refactorings of the reftable\n - code.\n + This is a more practical ordering when working on refactorings of the\n + reftable code.\n \n Signed-off-by: Han-Wen Nienhuys \n \n 9: 425e12d3667 = 9: 0abaab1d7c4 reftable: drop stray printf in readwrite_test\n 10: 67d858ec59e = 10: 004b7a736bf reftable: handle null refnames in reftable_ref_record_equal\n 11: 1e50924894f = 11: 951efcebcd2 reftable: make reftable-record.h function signatures const correct\n 12: c81b17ad57e = 12: 6385e449ba7 reftable: implement record equality generically\n 13: 4175089ec43 = 13: 69c4a554882 reftable: remove outdated file reftable.c\n 14: ede2e792ab6 ! 14: 54bcac325bd reftable: make reftable_record a tagged union\n @@ Metadata\n ## Commit message ##\n reftable: make reftable_record a tagged union\n \n - This reduces the amount of glue code, because we don't need a void pointer or\n - vtable within the structure.\n + This reduces the amount of glue code, because we don't need a void\n + pointer or vtable within the structure.\n \n - The only snag is that reftable_index_record contain a strbuf, so it cannot be\n - zero-initialized. To address this, use reftable_new_record() to return fresh\n - instance, given a record type. Since reftable_new_record() doesn't cause heap\n - allocation anymore, it should be balanced with reftable_record_release() rather\n - than reftable_record_destroy().\n + The only snag is that reftable_index_record contain a strbuf, so it\n + cannot be zero-initialized. To address this, use reftable_new_record()\n + to return fresh instance, given a record type. Since\n + reftable_new_record() doesn't cause heap allocation anymore, it should\n + be balanced with reftable_record_release() rather than\n + reftable_record_destroy().\n \n Thanks to Peff for the suggestion.\n \n 15: 6c06bd91662 = 15: 82f140cab5c reftable: add print functions to the record types\n 16: e16bf0c5212 < -: ----------- reftable: be more paranoid about 0-length memcpy calls\n\n-- \ngitgitgadget\n"},{"id":"446563","messageId":"9218bd59b2570ab08f3a2e5c0b590656d00482c4.1642691534.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v6.git.git.1642691534.gitgitgadget@gmail.com","subject":"[PATCH v6 01/15] reftable: fix OOB stack write in print functions","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-20T15:12:00Z","receivedAt":"2022-01-20T15:12:20Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 6a5dac32dc6..8536bd03aa9 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -254,7 +254,7 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)\n void reftable_ref_record_print(struct reftable_ref_record *ref,\n \t\t\t uint32_t hash_id)\n {\n-\tchar hex[2 * GIT_SHA256_RAWSZ + 1] = { 0 }; /* BUG */\n+\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */\n \tprintf(\"ref{%s(%\" PRIu64 \") \", ref->refname, ref->update_index);\n \tswitch (ref->value_type) {\n \tcase REFTABLE_REF_SYMREF:\n@@ -586,7 +586,7 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {\n void reftable_log_record_print(struct reftable_log_record *log,\n \t\t\t uint32_t hash_id)\n {\n-\tchar hex[GIT_SHA256_RAWSZ + 1] = { 0 };\n+\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 };\n \n \tswitch (log->value_type) {\n \tcase REFTABLE_LOG_DELETION:\n-- \ngitgitgadget\n\n"},{"id":"446564","messageId":"315ce62e710e6e80dc8a4d959e5532b5f9142669.1642691534.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v6.git.git.1642691534.gitgitgadget@gmail.com","subject":"[PATCH v6 02/15] reftable: fix resource leak in block.c error path","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-20T15:12:01Z","receivedAt":"2022-01-20T15:12:22Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nAdd test coverage for corrupt zlib data. Fix memory leaks demonstrated by\nunittest.\n\nThis problem was discovered by a Coverity scan.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/block.c | 26 +++++++++------\n reftable/reader.c | 23 ++++++++------\n reftable/readwrite_test.c | 66 +++++++++++++++++++++++++++++++++++++++\n 3 files changed, 97 insertions(+), 18 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 855e3f5c947..6c8e8705205 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -188,13 +188,16 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tuint32_t full_block_size = table_block_size;\n \tuint8_t typ = block->data[header_off];\n \tuint32_t sz = get_be24(block->data + header_off + 1);\n-\n+\tint err = 0;\n \tuint16_t restart_count = 0;\n \tuint32_t restart_start = 0;\n \tuint8_t *restart_bytes = NULL;\n+\tuint8_t *uncompressed = NULL;\n \n-\tif (!reftable_is_block_type(typ))\n-\t\treturn REFTABLE_FORMAT_ERROR;\n+\tif (!reftable_is_block_type(typ)) {\n+\t\terr = REFTABLE_FORMAT_ERROR;\n+\t\tgoto done;\n+\t}\n \n \tif (typ == BLOCK_TYPE_LOG) {\n \t\tint block_header_skip = 4 + header_off;\n@@ -203,7 +206,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\tuLongf src_len = block->len - block_header_skip;\n \t\t/* Log blocks specify the *uncompressed* size in their header.\n \t\t */\n-\t\tuint8_t *uncompressed = reftable_malloc(sz);\n+\t\tuncompressed = reftable_malloc(sz);\n \n \t\t/* Copy over the block header verbatim. It's not compressed. */\n \t\tmemcpy(uncompressed, block->data, block_header_skip);\n@@ -212,16 +215,19 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\tif (Z_OK !=\n \t\t uncompress2(uncompressed + block_header_skip, &dst_len,\n \t\t\t\tblock->data + block_header_skip, &src_len)) {\n-\t\t\treftable_free(uncompressed);\n-\t\t\treturn REFTABLE_ZLIB_ERROR;\n+\t\t\terr = REFTABLE_ZLIB_ERROR;\n+\t\t\tgoto done;\n \t\t}\n \n-\t\tif (dst_len + block_header_skip != sz)\n-\t\t\treturn REFTABLE_FORMAT_ERROR;\n+\t\tif (dst_len + block_header_skip != sz) {\n+\t\t\terr = REFTABLE_FORMAT_ERROR;\n+\t\t\tgoto done;\n+\t\t}\n \n \t\t/* We're done with the input data. */\n \t\treftable_block_done(block);\n \t\tblock->data = uncompressed;\n+\t\tuncompressed = NULL;\n \t\tblock->len = sz;\n \t\tblock->source = malloc_block_source();\n \t\tfull_block_size = src_len + block_header_skip;\n@@ -251,7 +257,9 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tbr->restart_count = restart_count;\n \tbr->restart_bytes = restart_bytes;\n \n-\treturn 0;\n+done:\n+\treftable_free(uncompressed);\n+\treturn err;\n }\n \n static uint32_t block_reader_restart_offset(struct block_reader *br, int i)\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 006709a645a..35781593a29 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -290,28 +290,33 @@ int reader_init_block_reader(struct reftable_reader *r, struct block_reader *br,\n \n \terr = reader_get_block(r, &block, next_off, guess_block_size);\n \tif (err < 0)\n-\t\treturn err;\n+\t\tgoto done;\n \n \tblock_size = extract_block_size(block.data, &block_typ, next_off,\n \t\t\t\t\tr->version);\n-\tif (block_size < 0)\n-\t\treturn block_size;\n-\n+\tif (block_size < 0) {\n+\t\terr = block_size;\n+\t\tgoto done;\n+\t}\n \tif (want_typ != BLOCK_TYPE_ANY && block_typ != want_typ) {\n-\t\treftable_block_done(&block);\n-\t\treturn 1;\n+\t\terr = 1;\n+\t\tgoto done;\n \t}\n \n \tif (block_size > guess_block_size) {\n \t\treftable_block_done(&block);\n \t\terr = reader_get_block(r, &block, next_off, block_size);\n \t\tif (err < 0) {\n-\t\t\treturn err;\n+\t\t\tgoto done;\n \t\t}\n \t}\n \n-\treturn block_reader_init(br, &block, header_off, r->block_size,\n-\t\t\t\t hash_size(r->hash_id));\n+\terr = block_reader_init(br, &block, header_off, r->block_size,\n+\t\t\t\thash_size(r->hash_id));\n+done:\n+\treftable_block_done(&block);\n+\n+\treturn err;\n }\n \n static int table_iter_next_block(struct table_iter *dest,\ndiff --git a/reftable/readwrite_test.c b/reftable/readwrite_test.c\nindex 70c7aedba2c..edb9cfc1930 100644\n--- a/reftable/readwrite_test.c\n+++ b/reftable/readwrite_test.c\n@@ -288,6 +288,71 @@ static void test_log_write_read(void)\n \treader_close(&rd);\n }\n \n+static void test_log_zlib_corruption(void)\n+{\n+\tstruct reftable_write_options opts = {\n+\t\t.block_size = 256,\n+\t};\n+\tstruct reftable_iterator it = { 0 };\n+\tstruct reftable_reader rd = { 0 };\n+\tstruct reftable_block_source source = { 0 };\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tstruct reftable_writer *w =\n+\t\treftable_new_writer(&strbuf_add_void, &buf, &opts);\n+\tconst struct reftable_stats *stats = NULL;\n+\tuint8_t hash1[GIT_SHA1_RAWSZ] = { 1 };\n+\tuint8_t hash2[GIT_SHA1_RAWSZ] = { 2 };\n+\tchar message[100] = { 0 };\n+\tint err, i, n;\n+\n+\tstruct reftable_log_record log = {\n+\t\t.refname = \"refname\",\n+\t\t.value_type = REFTABLE_LOG_UPDATE,\n+\t\t.value = {\n+\t\t\t.update = {\n+\t\t\t\t.new_hash = hash1,\n+\t\t\t\t.old_hash = hash2,\n+\t\t\t\t.name = \"My Name\",\n+\t\t\t\t.email = \"myname@invalid\",\n+\t\t\t\t.message = message,\n+\t\t\t},\n+\t\t},\n+\t};\n+\n+\tfor (i = 0; i < sizeof(message) - 1; i++)\n+\t\tmessage[i] = (uint8_t)(rand() % 64 + ' ');\n+\n+\treftable_writer_set_limits(w, 1, 1);\n+\n+\terr = reftable_writer_add_log(w, &log);\n+\tEXPECT_ERR(err);\n+\n+\tn = reftable_writer_close(w);\n+\tEXPECT(n == 0);\n+\n+\tstats = writer_stats(w);\n+\tEXPECT(stats->log_stats.blocks > 0);\n+\treftable_writer_free(w);\n+\tw = NULL;\n+\n+\t/* corrupt the data. */\n+\tbuf.buf[50] ^= 0x99;\n+\n+\tblock_source_from_strbuf(&source, &buf);\n+\n+\terr = init_reader(&rd, &source, \"file.log\");\n+\tEXPECT_ERR(err);\n+\n+\terr = reftable_reader_seek_log(&rd, &it, \"refname\");\n+\tEXPECT(err == REFTABLE_ZLIB_ERROR);\n+\n+\treftable_iterator_destroy(&it);\n+\n+\t/* cleanup. */\n+\tstrbuf_release(&buf);\n+\treader_close(&rd);\n+}\n+\n static void test_table_read_write_sequential(void)\n {\n \tchar **names;\n@@ -667,6 +732,7 @@ static void test_corrupt_table(void)\n \n int readwrite_test_main(int argc, const char *argv[])\n {\n+\tRUN_TEST(test_log_zlib_corruption);\n \tRUN_TEST(test_corrupt_table);\n \tRUN_TEST(test_corrupt_table_empty);\n \tRUN_TEST(test_log_write_read);\n-- \ngitgitgadget\n\n"},{"id":"446565","messageId":"7cf65e141cfb6de7668a06a8d66cc6e254fac691.1642691534.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v6.git.git.1642691534.gitgitgadget@gmail.com","subject":"[PATCH v6 03/15] reftable: fix resource leak blocksource.c","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-20T15:12:02Z","receivedAt":"2022-01-20T15:12:23Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis would be triggered in the unlikely event of fstat() failing on an\nopened file.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/blocksource.c | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/reftable/blocksource.c b/reftable/blocksource.c\nindex 0044eecd9aa..2605371c28d 100644\n--- a/reftable/blocksource.c\n+++ b/reftable/blocksource.c\n@@ -134,8 +134,10 @@ int reftable_block_source_from_file(struct reftable_block_source *bs,\n \t}\n \n \terr = fstat(fd, &st);\n-\tif (err < 0)\n-\t\treturn -1;\n+\tif (err < 0) {\n+\t\tclose(fd);\n+\t\treturn REFTABLE_IO_ERROR;\n+\t}\n \n \tp = reftable_calloc(sizeof(struct file_block_source));\n \tp->size = st.st_size;\n-- \ngitgitgadget\n\n"},{"id":"446566","messageId":"8740ba92981c77f85b855e69c237fdb7449cca58.1642691534.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v6.git.git.1642691534.gitgitgadget@gmail.com","subject":"[PATCH v6 04/15] reftable: check reftable_stack_auto_compact() return value","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-20T15:12:03Z","receivedAt":"2022-01-20T15:12:26Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nFixes a problem detected by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack_test.c | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/reftable/stack_test.c b/reftable/stack_test.c\nindex f4c743db80c..e84f50d27ff 100644\n--- a/reftable/stack_test.c\n+++ b/reftable/stack_test.c\n@@ -839,6 +839,7 @@ static void test_reftable_stack_auto_compaction(void)\n \t\tEXPECT_ERR(err);\n \n \t\terr = reftable_stack_auto_compact(st);\n+\t\tEXPECT_ERR(err);\n \t\tEXPECT(i < 3 || st->merged->stack_len < 2 * fastlog2(i));\n \t}\n \n-- \ngitgitgadget\n\n"},{"id":"446567","messageId":"630f67bdb519251815f878fba73b07dd0ae75320.1642691534.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v6.git.git.1642691534.gitgitgadget@gmail.com","subject":"[PATCH v6 06/15] reftable: fix resource warning","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-20T15:12:05Z","receivedAt":"2022-01-20T15:12:28Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis would trigger in the unlikely event that we are compacting, and\nthe next available file handle is 0.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack.c | 10 +++++-----\n 1 file changed, 5 insertions(+), 5 deletions(-)\n\ndiff --git a/reftable/stack.c b/reftable/stack.c\nindex 56bf5f2d84a..ddbdf1b9c8b 100644\n--- a/reftable/stack.c\n+++ b/reftable/stack.c\n@@ -889,7 +889,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \tstruct strbuf new_table_path = STRBUF_INIT;\n \tint err = 0;\n \tint have_lock = 0;\n-\tint lock_file_fd = 0;\n+\tint lock_file_fd = -1;\n \tint compact_count = last - first + 1;\n \tchar **listp = NULL;\n \tchar **delete_on_success =\n@@ -923,7 +923,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \t}\n \t/* Don't want to write to the lock for now. */\n \tclose(lock_file_fd);\n-\tlock_file_fd = 0;\n+\tlock_file_fd = -1;\n \n \thave_lock = 1;\n \terr = stack_uptodate(st);\n@@ -1031,7 +1031,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \t\tgoto done;\n \t}\n \terr = close(lock_file_fd);\n-\tlock_file_fd = 0;\n+\tlock_file_fd = -1;\n \tif (err < 0) {\n \t\terr = REFTABLE_IO_ERROR;\n \t\tunlink(new_table_path.buf);\n@@ -1068,9 +1068,9 @@ done:\n \t\tlistp++;\n \t}\n \tfree_names(subtable_locks);\n-\tif (lock_file_fd > 0) {\n+\tif (lock_file_fd >= 0) {\n \t\tclose(lock_file_fd);\n-\t\tlock_file_fd = 0;\n+\t\tlock_file_fd = -1;\n \t}\n \tif (have_lock) {\n \t\tunlink(lock_file_name.buf);\n-- \ngitgitgadget\n\n"},{"id":"446568","messageId":"08be6d90a4890b63fe9b0885af0df084c9ca81fd.1642691534.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v6.git.git.1642691534.gitgitgadget@gmail.com","subject":"[PATCH v6 05/15] reftable: ignore remove() return value in stack_test.c","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-20T15:12:04Z","receivedAt":"2022-01-20T15:12:29Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nIf the cleanup fails, there is nothing we can do.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack_test.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/reftable/stack_test.c b/reftable/stack_test.c\nindex e84f50d27ff..19fe4e20085 100644\n--- a/reftable/stack_test.c\n+++ b/reftable/stack_test.c\n@@ -90,7 +90,7 @@ static void test_read_file(void)\n \t\tEXPECT(0 == strcmp(want[i], names[i]));\n \t}\n \tfree_names(names);\n-\tremove(fn);\n+\t(void) remove(fn);\n }\n \n static void test_parse_names(void)\n-- \ngitgitgadget\n\n"},{"id":"446569","messageId":"afee67fb60d28a1672601bf5c5dafef01f4ee1b8.1642691534.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v6.git.git.1642691534.gitgitgadget@gmail.com","subject":"[PATCH v6 07/15] reftable: all xxx_free() functions accept NULL arguments","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-20T15:12:06Z","receivedAt":"2022-01-20T15:12:30Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis fixes NULL derefs in error paths. Spotted by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/reader.c | 2 ++\n reftable/writer.c | 2 ++\n 2 files changed, 4 insertions(+)\n\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 35781593a29..272378ed1d5 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -646,6 +646,8 @@ int reftable_new_reader(struct reftable_reader **p,\n \n void reftable_reader_free(struct reftable_reader *r)\n {\n+\tif (!r)\n+\t\treturn;\n \treader_close(r);\n \treftable_free(r);\n }\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex 35c8649c9b7..83a23daf60b 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -150,6 +150,8 @@ void reftable_writer_set_limits(struct reftable_writer *w, uint64_t min,\n \n void reftable_writer_free(struct reftable_writer *w)\n {\n+\tif (!w)\n+\t\treturn;\n \treftable_free(w->block);\n \treftable_free(w);\n }\n-- \ngitgitgadget\n\n"},{"id":"446570","messageId":"0abaab1d7c47dfcea4490459897dd100d2be39a5.1642691534.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v6.git.git.1642691534.gitgitgadget@gmail.com","subject":"[PATCH v6 09/15] reftable: drop stray printf in readwrite_test","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-20T15:12:08Z","receivedAt":"2022-01-20T15:12:31Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/readwrite_test.c | 1 -\n 1 file changed, 1 deletion(-)\n\ndiff --git a/reftable/readwrite_test.c b/reftable/readwrite_test.c\nindex edb9cfc1930..605ba0f9fd4 100644\n--- a/reftable/readwrite_test.c\n+++ b/reftable/readwrite_test.c\n@@ -696,7 +696,6 @@ static void test_write_key_order(void)\n \terr = reftable_writer_add_ref(w, &refs[0]);\n \tEXPECT_ERR(err);\n \terr = reftable_writer_add_ref(w, &refs[1]);\n-\tprintf(\"%d\\n\", err);\n \tEXPECT(err == REFTABLE_API_ERROR);\n \treftable_writer_close(w);\n \treftable_writer_free(w);\n-- \ngitgitgadget\n\n"},{"id":"446571","messageId":"004b7a736bf5ed847bad6518a5a9434c6176f98a.1642691534.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v6.git.git.1642691534.gitgitgadget@gmail.com","subject":"[PATCH v6 10/15] reftable: handle null refnames in reftable_ref_record_equal","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-20T15:12:09Z","receivedAt":"2022-01-20T15:12:33Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSpotted by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 8536bd03aa9..8bbcbff1e69 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -1154,9 +1154,11 @@ int reftable_ref_record_equal(struct reftable_ref_record *a,\n \t\t\t struct reftable_ref_record *b, int hash_size)\n {\n \tassert(hash_size > 0);\n-\tif (!(0 == strcmp(a->refname, b->refname) &&\n-\t a->update_index == b->update_index &&\n-\t a->value_type == b->value_type))\n+\tif (!null_streq(a->refname, b->refname))\n+\t\treturn 0;\n+\n+\tif (a->update_index != b->update_index ||\n+\t a->value_type != b->value_type)\n \t\treturn 0;\n \n \tswitch (a->value_type) {\n-- \ngitgitgadget\n\n"},{"id":"446572","messageId":"6f4f24c17ad91b28b22deff2530039bcfde2cbe9.1642691534.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v6.git.git.1642691534.gitgitgadget@gmail.com","subject":"[PATCH v6 08/15] reftable: order unittests by complexity","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-20T15:12:07Z","receivedAt":"2022-01-20T15:12:35Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis is a more practical ordering when working on refactorings of the\nreftable code.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n t/helper/test-reftable.c | 9 +++++----\n 1 file changed, 5 insertions(+), 4 deletions(-)\n\ndiff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c\nindex 26b03d7b789..1f0a28cbb64 100644\n--- a/t/helper/test-reftable.c\n+++ b/t/helper/test-reftable.c\n@@ -3,15 +3,16 @@\n \n int cmd__reftable(int argc, const char **argv)\n {\n+\t/* test from simple to complex. */\n \tbasics_test_main(argc, argv);\n+\trecord_test_main(argc, argv);\n \tblock_test_main(argc, argv);\n-\tmerged_test_main(argc, argv);\n+\ttree_test_main(argc, argv);\n \tpq_test_main(argc, argv);\n-\trecord_test_main(argc, argv);\n-\trefname_test_main(argc, argv);\n \treadwrite_test_main(argc, argv);\n+\tmerged_test_main(argc, argv);\n \tstack_test_main(argc, argv);\n-\ttree_test_main(argc, argv);\n+\trefname_test_main(argc, argv);\n \treturn 0;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"446573","messageId":"951efcebcd2097ec3af71703290c9e8c293bef05.1642691534.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v6.git.git.1642691534.gitgitgadget@gmail.com","subject":"[PATCH v6 11/15] reftable: make reftable-record.h function signatures const correct","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-20T15:12:10Z","receivedAt":"2022-01-20T15:12:36Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 14 +++++++-------\n reftable/reftable-record.h | 14 +++++++-------\n 2 files changed, 14 insertions(+), 14 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 8bbcbff1e69..f7c77c51539 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -72,7 +72,7 @@ int reftable_is_block_type(uint8_t typ)\n \treturn 0;\n }\n \n-uint8_t *reftable_ref_record_val1(struct reftable_ref_record *rec)\n+uint8_t *reftable_ref_record_val1(const struct reftable_ref_record *rec)\n {\n \tswitch (rec->value_type) {\n \tcase REFTABLE_REF_VAL1:\n@@ -84,7 +84,7 @@ uint8_t *reftable_ref_record_val1(struct reftable_ref_record *rec)\n \t}\n }\n \n-uint8_t *reftable_ref_record_val2(struct reftable_ref_record *rec)\n+uint8_t *reftable_ref_record_val2(const struct reftable_ref_record *rec)\n {\n \tswitch (rec->value_type) {\n \tcase REFTABLE_REF_VAL2:\n@@ -251,7 +251,7 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)\n \t}\n }\n \n-void reftable_ref_record_print(struct reftable_ref_record *ref,\n+void reftable_ref_record_print(const struct reftable_ref_record *ref,\n \t\t\t uint32_t hash_id)\n {\n \tchar hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */\n@@ -881,8 +881,8 @@ static int zero_hash_eq(uint8_t *a, uint8_t *b, int sz)\n \treturn !memcmp(a, b, sz);\n }\n \n-int reftable_log_record_equal(struct reftable_log_record *a,\n-\t\t\t struct reftable_log_record *b, int hash_size)\n+int reftable_log_record_equal(const struct reftable_log_record *a,\n+\t\t\t const struct reftable_log_record *b, int hash_size)\n {\n \tif (!(null_streq(a->refname, b->refname) &&\n \t a->update_index == b->update_index &&\n@@ -1150,8 +1150,8 @@ static int hash_equal(uint8_t *a, uint8_t *b, int hash_size)\n \treturn a == b;\n }\n \n-int reftable_ref_record_equal(struct reftable_ref_record *a,\n-\t\t\t struct reftable_ref_record *b, int hash_size)\n+int reftable_ref_record_equal(const struct reftable_ref_record *a,\n+\t\t\t const struct reftable_ref_record *b, int hash_size)\n {\n \tassert(hash_size > 0);\n \tif (!null_streq(a->refname, b->refname))\ndiff --git a/reftable/reftable-record.h b/reftable/reftable-record.h\nindex 5370d2288c7..67104f8fbfe 100644\n--- a/reftable/reftable-record.h\n+++ b/reftable/reftable-record.h\n@@ -49,25 +49,25 @@ struct reftable_ref_record {\n \n /* Returns the first hash, or NULL if `rec` is not of type\n * REFTABLE_REF_VAL1 or REFTABLE_REF_VAL2. */\n-uint8_t *reftable_ref_record_val1(struct reftable_ref_record *rec);\n+uint8_t *reftable_ref_record_val1(const struct reftable_ref_record *rec);\n \n /* Returns the second hash, or NULL if `rec` is not of type\n * REFTABLE_REF_VAL2. */\n-uint8_t *reftable_ref_record_val2(struct reftable_ref_record *rec);\n+uint8_t *reftable_ref_record_val2(const struct reftable_ref_record *rec);\n \n /* returns whether 'ref' represents a deletion */\n int reftable_ref_record_is_deletion(const struct reftable_ref_record *ref);\n \n /* prints a reftable_ref_record onto stdout. Useful for debugging. */\n-void reftable_ref_record_print(struct reftable_ref_record *ref,\n+void reftable_ref_record_print(const struct reftable_ref_record *ref,\n \t\t\t uint32_t hash_id);\n \n /* frees and nulls all pointer values inside `ref`. */\n void reftable_ref_record_release(struct reftable_ref_record *ref);\n \n /* returns whether two reftable_ref_records are the same. Useful for testing. */\n-int reftable_ref_record_equal(struct reftable_ref_record *a,\n-\t\t\t struct reftable_ref_record *b, int hash_size);\n+int reftable_ref_record_equal(const struct reftable_ref_record *a,\n+\t\t\t const struct reftable_ref_record *b, int hash_size);\n \n /* reftable_log_record holds a reflog entry */\n struct reftable_log_record {\n@@ -104,8 +104,8 @@ int reftable_log_record_is_deletion(const struct reftable_log_record *log);\n void reftable_log_record_release(struct reftable_log_record *log);\n \n /* returns whether two records are equal. Useful for testing. */\n-int reftable_log_record_equal(struct reftable_log_record *a,\n-\t\t\t struct reftable_log_record *b, int hash_size);\n+int reftable_log_record_equal(const struct reftable_log_record *a,\n+\t\t\t const struct reftable_log_record *b, int hash_size);\n \n /* dumps a reftable_log_record on stdout, for debugging/testing. */\n void reftable_log_record_print(struct reftable_log_record *log,\n-- \ngitgitgadget\n\n"},{"id":"446574","messageId":"6385e449ba714e3ce41f10276fc60e9757519492.1642691534.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v6.git.git.1642691534.gitgitgadget@gmail.com","subject":"[PATCH v6 12/15] reftable: implement record equality generically","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-20T15:12:11Z","receivedAt":"2022-01-20T15:12:37Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis simplifies unittests a little, and provides further coverage for\nreftable_record_copy().\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 57 +++++++++++++++++++++++++++++++++++++++++-\n reftable/record.h | 5 +++-\n reftable/record_test.c | 23 +++--------------\n 3 files changed, 63 insertions(+), 22 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex f7c77c51539..2a9e41a992e 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -430,6 +430,15 @@ static int reftable_ref_record_is_deletion_void(const void *p)\n \t\t(const struct reftable_ref_record *)p);\n }\n \n+\n+static int reftable_ref_record_equal_void(const void *a,\n+\t\t\t\t\t const void *b, int hash_size)\n+{\n+\tstruct reftable_ref_record *ra = (struct reftable_ref_record *) a;\n+\tstruct reftable_ref_record *rb = (struct reftable_ref_record *) b;\n+\treturn reftable_ref_record_equal(ra, rb, hash_size);\n+}\n+\n static struct reftable_record_vtable reftable_ref_record_vtable = {\n \t.key = &reftable_ref_record_key,\n \t.type = BLOCK_TYPE_REF,\n@@ -439,6 +448,7 @@ static struct reftable_record_vtable reftable_ref_record_vtable = {\n \t.decode = &reftable_ref_record_decode,\n \t.release = &reftable_ref_record_release_void,\n \t.is_deletion = &reftable_ref_record_is_deletion_void,\n+\t.equal = &reftable_ref_record_equal_void,\n };\n \n static void reftable_obj_record_key(const void *r, struct strbuf *dest)\n@@ -572,6 +582,25 @@ static int not_a_deletion(const void *p)\n \treturn 0;\n }\n \n+static int reftable_obj_record_equal_void(const void *a, const void *b, int hash_size)\n+{\n+\tstruct reftable_obj_record *ra = (struct reftable_obj_record *) a;\n+\tstruct reftable_obj_record *rb = (struct reftable_obj_record *) b;\n+\n+\tif (ra->hash_prefix_len != rb->hash_prefix_len\n+\t || ra->offset_len != rb->offset_len)\n+\t\treturn 0;\n+\n+\tif (ra->hash_prefix_len &&\n+\t memcmp(ra->hash_prefix, rb->hash_prefix, ra->hash_prefix_len))\n+\t\treturn 0;\n+\tif (ra->offset_len &&\n+\t memcmp(ra->offsets, rb->offsets, ra->offset_len * sizeof(uint64_t)))\n+\t\treturn 0;\n+\n+\treturn 1;\n+}\n+\n static struct reftable_record_vtable reftable_obj_record_vtable = {\n \t.key = &reftable_obj_record_key,\n \t.type = BLOCK_TYPE_OBJ,\n@@ -580,7 +609,8 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {\n \t.encode = &reftable_obj_record_encode,\n \t.decode = &reftable_obj_record_decode,\n \t.release = &reftable_obj_record_release,\n-\t.is_deletion = not_a_deletion,\n+\t.is_deletion = ¬_a_deletion,\n+\t.equal = &reftable_obj_record_equal_void,\n };\n \n void reftable_log_record_print(struct reftable_log_record *log,\n@@ -881,6 +911,14 @@ static int zero_hash_eq(uint8_t *a, uint8_t *b, int sz)\n \treturn !memcmp(a, b, sz);\n }\n \n+static int reftable_log_record_equal_void(const void *a,\n+\t\t\t\t\t const void *b, int hash_size)\n+{\n+\treturn reftable_log_record_equal((struct reftable_log_record *) a,\n+\t\t\t\t\t (struct reftable_log_record *) b,\n+\t\t\t\t\t hash_size);\n+}\n+\n int reftable_log_record_equal(const struct reftable_log_record *a,\n \t\t\t const struct reftable_log_record *b, int hash_size)\n {\n@@ -924,6 +962,7 @@ static struct reftable_record_vtable reftable_log_record_vtable = {\n \t.decode = &reftable_log_record_decode,\n \t.release = &reftable_log_record_release_void,\n \t.is_deletion = &reftable_log_record_is_deletion_void,\n+\t.equal = &reftable_log_record_equal_void\n };\n \n struct reftable_record reftable_new_record(uint8_t typ)\n@@ -1042,6 +1081,14 @@ static int reftable_index_record_decode(void *rec, struct strbuf key,\n \treturn start.len - in.len;\n }\n \n+static int reftable_index_record_equal(const void *a, const void *b, int hash_size)\n+{\n+\tstruct reftable_index_record *ia = (struct reftable_index_record *) a;\n+\tstruct reftable_index_record *ib = (struct reftable_index_record *) b;\n+\n+\treturn ia->offset == ib->offset && !strbuf_cmp(&ia->last_key, &ib->last_key);\n+}\n+\n static struct reftable_record_vtable reftable_index_record_vtable = {\n \t.key = &reftable_index_record_key,\n \t.type = BLOCK_TYPE_INDEX,\n@@ -1051,6 +1098,7 @@ static struct reftable_record_vtable reftable_index_record_vtable = {\n \t.decode = &reftable_index_record_decode,\n \t.release = &reftable_index_record_release,\n \t.is_deletion = ¬_a_deletion,\n+\t.equal = &reftable_index_record_equal,\n };\n \n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)\n@@ -1098,6 +1146,13 @@ int reftable_record_is_deletion(struct reftable_record *rec)\n \treturn rec->ops->is_deletion(rec->data);\n }\n \n+int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size)\n+{\n+\tif (a->ops != b->ops)\n+\t\treturn 0;\n+\treturn a->ops->equal(a->data, b->data, hash_size);\n+}\n+\n void reftable_record_from_ref(struct reftable_record *rec,\n \t\t\t struct reftable_ref_record *ref_rec)\n {\ndiff --git a/reftable/record.h b/reftable/record.h\nindex 498e8c50bf4..da75d7d1f11 100644\n--- a/reftable/record.h\n+++ b/reftable/record.h\n@@ -58,6 +58,9 @@ struct reftable_record_vtable {\n \n \t/* is this a tombstone? */\n \tint (*is_deletion)(const void *rec);\n+\n+\t/* Are two records equal? This assumes they have the same type. Returns 0 for non-equal. */\n+\tint (*equal)(const void *a, const void *b, int hash_size);\n };\n \n /* record is a generic wrapper for different types of records. */\n@@ -98,7 +101,7 @@ struct reftable_obj_record {\n };\n \n /* see struct record_vtable */\n-\n+int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size);\n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest);\n uint8_t reftable_record_type(struct reftable_record *rec);\n void reftable_record_copy_from(struct reftable_record *rec,\ndiff --git a/reftable/record_test.c b/reftable/record_test.c\nindex f4ad7cace41..92680848156 100644\n--- a/reftable/record_test.c\n+++ b/reftable/record_test.c\n@@ -21,18 +21,7 @@ static void test_copy(struct reftable_record *rec)\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \t/* do it twice to catch memory leaks */\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n-\tswitch (reftable_record_type(©)) {\n-\tcase BLOCK_TYPE_REF:\n-\t\tEXPECT(reftable_ref_record_equal(reftable_record_as_ref(©),\n-\t\t\t\t\t\t reftable_record_as_ref(rec),\n-\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n-\t\tbreak;\n-\tcase BLOCK_TYPE_LOG:\n-\t\tEXPECT(reftable_log_record_equal(reftable_record_as_log(©),\n-\t\t\t\t\t\t reftable_record_as_log(rec),\n-\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n-\t\tbreak;\n-\t}\n+\tEXPECT(reftable_record_equal(rec, ©, GIT_SHA1_RAWSZ));\n \treftable_record_destroy(©);\n }\n \n@@ -346,13 +335,7 @@ static void test_reftable_obj_record_roundtrip(void)\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(in.hash_prefix_len == out.hash_prefix_len);\n-\t\tEXPECT(in.offset_len == out.offset_len);\n-\n-\t\tEXPECT(!memcmp(in.hash_prefix, out.hash_prefix,\n-\t\t\t in.hash_prefix_len));\n-\t\tEXPECT(0 == memcmp(in.offsets, out.offsets,\n-\t\t\t\t sizeof(uint64_t) * in.offset_len));\n+\t\tEXPECT(reftable_record_equal(&rec, &rec_out, GIT_SHA1_RAWSZ));\n \t\tstrbuf_release(&key);\n \t\treftable_record_release(&rec_out);\n \t}\n@@ -390,7 +373,7 @@ static void test_reftable_index_record_roundtrip(void)\n \tm = reftable_record_decode(&out_rec, key, extra, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(m == n);\n \n-\tEXPECT(in.offset == out.offset);\n+\tEXPECT(reftable_record_equal(&rec, &out_rec, GIT_SHA1_RAWSZ));\n \n \treftable_record_release(&out_rec);\n \tstrbuf_release(&key);\n-- \ngitgitgadget\n\n"},{"id":"446575","messageId":"82f140cab5c30b59de534d773a5d00343e0868f9.1642691534.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v6.git.git.1642691534.gitgitgadget@gmail.com","subject":"[PATCH v6 15/15] reftable: add print functions to the record types","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-20T15:12:14Z","receivedAt":"2022-01-20T15:12:39Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis isn't used per se, but it is useful for debugging, especially\nWindows CI failures.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 85 +++++++++++++++++++++++++++++++++++-------\n reftable/record.h | 4 ++\n reftable/record_test.c | 21 ++++++++++-\n 3 files changed, 95 insertions(+), 15 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex a8cee628942..fbaa1fbef56 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -255,8 +255,8 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)\n \t}\n }\n \n-void reftable_ref_record_print(const struct reftable_ref_record *ref,\n-\t\t\t uint32_t hash_id)\n+static void reftable_ref_record_print_sz(const struct reftable_ref_record *ref,\n+\t\t\t\t\t int hash_size)\n {\n \tchar hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */\n \tprintf(\"ref{%s(%\" PRIu64 \") \", ref->refname, ref->update_index);\n@@ -265,14 +265,14 @@ void reftable_ref_record_print(const struct reftable_ref_record *ref,\n \t\tprintf(\"=> %s\", ref->value.symref);\n \t\tbreak;\n \tcase REFTABLE_REF_VAL2:\n-\t\thex_format(hex, ref->value.val2.value, hash_size(hash_id));\n+\t\thex_format(hex, ref->value.val2.value, hash_size);\n \t\tprintf(\"val 2 %s\", hex);\n \t\thex_format(hex, ref->value.val2.target_value,\n-\t\t\t hash_size(hash_id));\n+\t\t\t hash_size);\n \t\tprintf(\"(T %s)\", hex);\n \t\tbreak;\n \tcase REFTABLE_REF_VAL1:\n-\t\thex_format(hex, ref->value.val1, hash_size(hash_id));\n+\t\thex_format(hex, ref->value.val1, hash_size);\n \t\tprintf(\"val 1 %s\", hex);\n \t\tbreak;\n \tcase REFTABLE_REF_DELETION:\n@@ -282,6 +282,11 @@ void reftable_ref_record_print(const struct reftable_ref_record *ref,\n \tprintf(\"}\\n\");\n }\n \n+void reftable_ref_record_print(const struct reftable_ref_record *ref,\n+\t\t\t uint32_t hash_id) {\n+\treftable_ref_record_print_sz(ref, hash_size(hash_id));\n+}\n+\n static void reftable_ref_record_release_void(void *rec)\n {\n \treftable_ref_record_release(rec);\n@@ -443,6 +448,12 @@ static int reftable_ref_record_equal_void(const void *a,\n \treturn reftable_ref_record_equal(ra, rb, hash_size);\n }\n \n+static void reftable_ref_record_print_void(const void *rec,\n+\t\t\t\t\t int hash_size)\n+{\n+\treftable_ref_record_print_sz((struct reftable_ref_record *) rec, hash_size);\n+}\n+\n static struct reftable_record_vtable reftable_ref_record_vtable = {\n \t.key = &reftable_ref_record_key,\n \t.type = BLOCK_TYPE_REF,\n@@ -453,6 +464,7 @@ static struct reftable_record_vtable reftable_ref_record_vtable = {\n \t.release = &reftable_ref_record_release_void,\n \t.is_deletion = &reftable_ref_record_is_deletion_void,\n \t.equal = &reftable_ref_record_equal_void,\n+\t.print = &reftable_ref_record_print_void,\n };\n \n static void reftable_obj_record_key(const void *r, struct strbuf *dest)\n@@ -471,6 +483,21 @@ static void reftable_obj_record_release(void *rec)\n \tmemset(obj, 0, sizeof(struct reftable_obj_record));\n }\n \n+static void reftable_obj_record_print(const void *rec, int hash_size)\n+{\n+\tconst struct reftable_obj_record *obj = rec;\n+\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 };\n+\tstruct strbuf offset_str = STRBUF_INIT;\n+\tint i;\n+\n+\tfor (i = 0; i < obj->offset_len; i++)\n+\t\tstrbuf_addf(&offset_str, \"%\" PRIu64 \" \", obj->offsets[i]);\n+\thex_format(hex, obj->hash_prefix, obj->hash_prefix_len);\n+\tprintf(\"prefix %s (len %d), offsets [%s]\\n\",\n+\t hex, obj->hash_prefix_len, offset_str.buf);\n+\tstrbuf_release(&offset_str);\n+}\n+\n static void reftable_obj_record_copy_from(void *rec, const void *src_rec,\n \t\t\t\t\t int hash_size)\n {\n@@ -617,31 +644,41 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {\n \t.release = &reftable_obj_record_release,\n \t.is_deletion = ¬_a_deletion,\n \t.equal = &reftable_obj_record_equal_void,\n+\t.print = &reftable_obj_record_print,\n };\n \n-void reftable_log_record_print(struct reftable_log_record *log,\n-\t\t\t uint32_t hash_id)\n+static void reftable_log_record_print_sz(struct reftable_log_record *log,\n+\t\t\t\t\t int hash_size)\n {\n \tchar hex[GIT_MAX_HEXSZ + 1] = { 0 };\n \n \tswitch (log->value_type) {\n \tcase REFTABLE_LOG_DELETION:\n-\t\tprintf(\"log{%s(%\" PRIu64 \") delete\", log->refname,\n+\t\tprintf(\"log{%s(%\" PRIu64 \") delete\\n\", log->refname,\n \t\t log->update_index);\n \t\tbreak;\n \tcase REFTABLE_LOG_UPDATE:\n \t\tprintf(\"log{%s(%\" PRIu64 \") %s <%s> %\" PRIu64 \" %04d\\n\",\n-\t\t log->refname, log->update_index, log->value.update.name,\n-\t\t log->value.update.email, log->value.update.time,\n+\t\t log->refname, log->update_index,\n+\t\t log->value.update.name ? log->value.update.name : \"\",\n+\t\t log->value.update.email ? log->value.update.email : \"\",\n+\t\t log->value.update.time,\n \t\t log->value.update.tz_offset);\n-\t\thex_format(hex, log->value.update.old_hash, hash_size(hash_id));\n+\t\thex_format(hex, log->value.update.old_hash, hash_size);\n \t\tprintf(\"%s => \", hex);\n-\t\thex_format(hex, log->value.update.new_hash, hash_size(hash_id));\n-\t\tprintf(\"%s\\n\\n%s\\n}\\n\", hex, log->value.update.message);\n+\t\thex_format(hex, log->value.update.new_hash, hash_size);\n+\t\tprintf(\"%s\\n\\n%s\\n}\\n\", hex,\n+\t\t log->value.update.message ? log->value.update.message : \"\");\n \t\tbreak;\n \t}\n }\n \n+void reftable_log_record_print(struct reftable_log_record *log,\n+\t\t\t\t uint32_t hash_id)\n+{\n+\treftable_log_record_print_sz(log, hash_size(hash_id));\n+}\n+\n static void reftable_log_record_key(const void *r, struct strbuf *dest)\n {\n \tconst struct reftable_log_record *rec =\n@@ -959,6 +996,11 @@ static int reftable_log_record_is_deletion_void(const void *p)\n \t\t(const struct reftable_log_record *)p);\n }\n \n+static void reftable_log_record_print_void(const void *rec, int hash_size)\n+{\n+\treftable_log_record_print_sz((struct reftable_log_record*)rec, hash_size);\n+}\n+\n static struct reftable_record_vtable reftable_log_record_vtable = {\n \t.key = &reftable_log_record_key,\n \t.type = BLOCK_TYPE_LOG,\n@@ -968,7 +1010,8 @@ static struct reftable_record_vtable reftable_log_record_vtable = {\n \t.decode = &reftable_log_record_decode,\n \t.release = &reftable_log_record_release_void,\n \t.is_deletion = &reftable_log_record_is_deletion_void,\n-\t.equal = &reftable_log_record_equal_void\n+\t.equal = &reftable_log_record_equal_void,\n+\t.print = &reftable_log_record_print_void,\n };\n \n static void reftable_index_record_key(const void *r, struct strbuf *dest)\n@@ -1043,6 +1086,13 @@ static int reftable_index_record_equal(const void *a, const void *b, int hash_si\n \treturn ia->offset == ib->offset && !strbuf_cmp(&ia->last_key, &ib->last_key);\n }\n \n+static void reftable_index_record_print(const void *rec, int hash_size)\n+{\n+\tconst struct reftable_index_record *idx = rec;\n+\t/* TODO: escape null chars? */\n+\tprintf(\"\\\"%s\\\" %\" PRIu64 \"\\n\", idx->last_key.buf, idx->offset);\n+}\n+\n static struct reftable_record_vtable reftable_index_record_vtable = {\n \t.key = &reftable_index_record_key,\n \t.type = BLOCK_TYPE_INDEX,\n@@ -1053,6 +1103,7 @@ static struct reftable_record_vtable reftable_index_record_vtable = {\n \t.release = &reftable_index_record_release,\n \t.is_deletion = ¬_a_deletion,\n \t.equal = &reftable_index_record_equal,\n+\t.print = &reftable_index_record_print,\n };\n \n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)\n@@ -1255,3 +1306,9 @@ struct reftable_record reftable_new_record(uint8_t typ)\n \t}\n \treturn clean;\n }\n+\n+void reftable_record_print(struct reftable_record *rec, int hash_size)\n+{\n+\tprintf(\"'%c': \", rec->type);\n+\treftable_record_vtable(rec)->print(reftable_record_data(rec), hash_size);\n+}\ndiff --git a/reftable/record.h b/reftable/record.h\nindex 010a322e901..fd80cd451d5 100644\n--- a/reftable/record.h\n+++ b/reftable/record.h\n@@ -61,6 +61,9 @@ struct reftable_record_vtable {\n \n \t/* Are two records equal? This assumes they have the same type. Returns 0 for non-equal. */\n \tint (*equal)(const void *a, const void *b, int hash_size);\n+\n+\t/* Print on stdout, for debugging. */\n+\tvoid (*print)(const void *rec, int hash_size);\n };\n \n /* returns true for recognized block types. Block start with the block type. */\n@@ -112,6 +115,7 @@ struct reftable_record {\n \n /* see struct record_vtable */\n int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size);\n+void reftable_record_print(struct reftable_record *rec, int hash_size);\n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest);\n uint8_t reftable_record_type(struct reftable_record *rec);\n void reftable_record_copy_from(struct reftable_record *rec,\ndiff --git a/reftable/record_test.c b/reftable/record_test.c\nindex c6fdd1925a9..f91ea5e8830 100644\n--- a/reftable/record_test.c\n+++ b/reftable/record_test.c\n@@ -25,6 +25,10 @@ static void test_copy(struct reftable_record *rec)\n \t/* do it twice to catch memory leaks */\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \tEXPECT(reftable_record_equal(rec, ©, GIT_SHA1_RAWSZ));\n+\n+\tputs(\"testing print coverage:\\n\");\n+\treftable_record_print(©, GIT_SHA1_RAWSZ);\n+\n \treftable_record_release(©);\n }\n \n@@ -176,7 +180,8 @@ static void test_reftable_log_record_equal(void)\n static void test_reftable_log_record_roundtrip(void)\n {\n \tint i;\n-\tstruct reftable_log_record in[2] = {\n+\n+\tstruct reftable_log_record in[] = {\n \t\t{\n \t\t\t.refname = xstrdup(\"refs/heads/master\"),\n \t\t\t.update_index = 42,\n@@ -197,10 +202,24 @@ static void test_reftable_log_record_roundtrip(void)\n \t\t\t.refname = xstrdup(\"refs/heads/master\"),\n \t\t\t.update_index = 22,\n \t\t\t.value_type = REFTABLE_LOG_DELETION,\n+\t\t},\n+\t\t{\n+\t\t\t.refname = xstrdup(\"branch\"),\n+\t\t\t.update_index = 33,\n+\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n+\t\t\t.value = {\n+\t\t\t\t.update = {\n+\t\t\t\t\t.old_hash = reftable_malloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t.new_hash = reftable_malloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t/* rest of fields left empty. */\n+\t\t\t\t},\n+\t\t\t},\n \t\t}\n \t};\n \tset_test_hash(in[0].value.update.new_hash, 1);\n \tset_test_hash(in[0].value.update.old_hash, 2);\n+\tset_test_hash(in[2].value.update.new_hash, 3);\n+\tset_test_hash(in[2].value.update.old_hash, 4);\n \tfor (i = 0; i < ARRAY_SIZE(in); i++) {\n \t\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG };\n \t\tstruct strbuf key = STRBUF_INIT;\n-- \ngitgitgadget\n"},{"id":"446576","messageId":"69c4a55488209f96ae605047ecd1709678518fa5.1642691534.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v6.git.git.1642691534.gitgitgadget@gmail.com","subject":"[PATCH v6 13/15] reftable: remove outdated file reftable.c","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-20T15:12:12Z","receivedAt":"2022-01-20T15:12:40Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis was renamed to generic.c, but the origin was never removed\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/reftable.c | 115 --------------------------------------------\n 1 file changed, 115 deletions(-)\n delete mode 100644 reftable/reftable.c\n\ndiff --git a/reftable/reftable.c b/reftable/reftable.c\ndeleted file mode 100644\nindex 0e4607a7cd6..00000000000\n--- a/reftable/reftable.c\n+++ /dev/null\n@@ -1,115 +0,0 @@\n-/*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n-\n-#include \"basics.h\"\n-#include \"record.h\"\n-#include \"generic.h\"\n-#include \"reftable-iterator.h\"\n-#include \"reftable-generic.h\"\n-\n-int reftable_table_seek_ref(struct reftable_table *tab,\n-\t\t\t struct reftable_iterator *it, const char *name)\n-{\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n-\treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n-}\n-\n-int reftable_table_read_ref(struct reftable_table *tab, const char *name,\n-\t\t\t struct reftable_ref_record *ref)\n-{\n-\tstruct reftable_iterator it = { NULL };\n-\tint err = reftable_table_seek_ref(tab, &it, name);\n-\tif (err)\n-\t\tgoto done;\n-\n-\terr = reftable_iterator_next_ref(&it, ref);\n-\tif (err)\n-\t\tgoto done;\n-\n-\tif (strcmp(ref->refname, name) ||\n-\t reftable_ref_record_is_deletion(ref)) {\n-\t\treftable_ref_record_release(ref);\n-\t\terr = 1;\n-\t\tgoto done;\n-\t}\n-\n-done:\n-\treftable_iterator_destroy(&it);\n-\treturn err;\n-}\n-\n-uint64_t reftable_table_max_update_index(struct reftable_table *tab)\n-{\n-\treturn tab->ops->max_update_index(tab->table_arg);\n-}\n-\n-uint64_t reftable_table_min_update_index(struct reftable_table *tab)\n-{\n-\treturn tab->ops->min_update_index(tab->table_arg);\n-}\n-\n-uint32_t reftable_table_hash_id(struct reftable_table *tab)\n-{\n-\treturn tab->ops->hash_id(tab->table_arg);\n-}\n-\n-void reftable_iterator_destroy(struct reftable_iterator *it)\n-{\n-\tif (!it->ops) {\n-\t\treturn;\n-\t}\n-\tit->ops->close(it->iter_arg);\n-\tit->ops = NULL;\n-\tFREE_AND_NULL(it->iter_arg);\n-}\n-\n-int reftable_iterator_next_ref(struct reftable_iterator *it,\n-\t\t\t struct reftable_ref_record *ref)\n-{\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, ref);\n-\treturn iterator_next(it, &rec);\n-}\n-\n-int reftable_iterator_next_log(struct reftable_iterator *it,\n-\t\t\t struct reftable_log_record *log)\n-{\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, log);\n-\treturn iterator_next(it, &rec);\n-}\n-\n-int iterator_next(struct reftable_iterator *it, struct reftable_record *rec)\n-{\n-\treturn it->ops->next(it->iter_arg, rec);\n-}\n-\n-static int empty_iterator_next(void *arg, struct reftable_record *rec)\n-{\n-\treturn 1;\n-}\n-\n-static void empty_iterator_close(void *arg)\n-{\n-}\n-\n-static struct reftable_iterator_vtable empty_vtable = {\n-\t.next = &empty_iterator_next,\n-\t.close = &empty_iterator_close,\n-};\n-\n-void iterator_set_empty(struct reftable_iterator *it)\n-{\n-\tassert(!it->ops);\n-\tit->iter_arg = NULL;\n-\tit->ops = &empty_vtable;\n-}\n-- \ngitgitgadget\n\n"},{"id":"446577","messageId":"54bcac325bd542aa81328e5616b2b7573d43adac.1642691534.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v6.git.git.1642691534.gitgitgadget@gmail.com","subject":"[PATCH v6 14/15] reftable: make reftable_record a tagged union","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-20T15:12:13Z","receivedAt":"2022-01-20T15:12:41Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis reduces the amount of glue code, because we don't need a void\npointer or vtable within the structure.\n\nThe only snag is that reftable_index_record contain a strbuf, so it\ncannot be zero-initialized. To address this, use reftable_new_record()\nto return fresh instance, given a record type. Since\nreftable_new_record() doesn't cause heap allocation anymore, it should\nbe balanced with reftable_record_release() rather than\nreftable_record_destroy().\n\nThanks to Peff for the suggestion.\n\nHelped-by: Jeff King \nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/block.c | 2 +-\n reftable/block_test.c | 22 ++---\n reftable/generic.c | 41 ++++----\n reftable/iter.c | 4 +-\n reftable/merged.c | 33 ++++---\n reftable/pq.c | 3 +-\n reftable/pq_test.c | 27 +++---\n reftable/reader.c | 87 +++++++++--------\n reftable/record.c | 212 +++++++++++++++++++----------------------\n reftable/record.h | 42 ++++----\n reftable/record_test.c | 157 +++++++++++++++---------------\n reftable/writer.c | 41 ++++----\n 12 files changed, 334 insertions(+), 337 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 6c8e8705205..2170748c5e9 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -421,7 +421,7 @@ int block_reader_seek(struct block_reader *br, struct block_iter *it,\n done:\n \tstrbuf_release(&key);\n \tstrbuf_release(&next.last_key);\n-\treftable_record_destroy(&rec);\n+\treftable_record_release(&rec);\n \n \treturn err;\n }\ndiff --git a/reftable/block_test.c b/reftable/block_test.c\nindex 4b3ea262dcb..fa2ee092ec0 100644\n--- a/reftable/block_test.c\n+++ b/reftable/block_test.c\n@@ -26,8 +26,9 @@ static void test_block_read_write(void)\n \tstruct block_writer bw = {\n \t\t.last_key = STRBUF_INIT,\n \t};\n-\tstruct reftable_ref_record ref = { NULL };\n-\tstruct reftable_record rec = { NULL };\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t};\n \tint i = 0;\n \tint n;\n \tstruct block_reader br = { 0 };\n@@ -40,7 +41,6 @@ static void test_block_read_write(void)\n \tblock.source = malloc_block_source();\n \tblock_writer_init(&bw, BLOCK_TYPE_REF, block.data, block_size,\n \t\t\t header_off, hash_size(GIT_SHA1_FORMAT_ID));\n-\treftable_record_from_ref(&rec, &ref);\n \n \tfor (i = 0; i < N; i++) {\n \t\tchar name[100];\n@@ -48,14 +48,14 @@ static void test_block_read_write(void)\n \t\tsnprintf(name, sizeof(name), \"branch%02d\", i);\n \t\tmemset(hash, i, sizeof(hash));\n \n-\t\tref.refname = name;\n-\t\tref.value_type = REFTABLE_REF_VAL1;\n-\t\tref.value.val1 = hash;\n+\t\trec.u.ref.refname = name;\n+\t\trec.u.ref.value_type = REFTABLE_REF_VAL1;\n+\t\trec.u.ref.value.val1 = hash;\n \n \t\tnames[i] = xstrdup(name);\n \t\tn = block_writer_add(&bw, &rec);\n-\t\tref.refname = NULL;\n-\t\tref.value_type = REFTABLE_REF_DELETION;\n+\t\trec.u.ref.refname = NULL;\n+\t\trec.u.ref.value_type = REFTABLE_REF_DELETION;\n \t\tEXPECT(n == 0);\n \t}\n \n@@ -74,7 +74,7 @@ static void test_block_read_write(void)\n \t\tif (r > 0) {\n \t\t\tbreak;\n \t\t}\n-\t\tEXPECT_STREQ(names[j], ref.refname);\n+\t\tEXPECT_STREQ(names[j], rec.u.ref.refname);\n \t\tj++;\n \t}\n \n@@ -92,7 +92,7 @@ static void test_block_read_write(void)\n \t\tn = block_iter_next(&it, &rec);\n \t\tEXPECT(n == 0);\n \n-\t\tEXPECT_STREQ(names[i], ref.refname);\n+\t\tEXPECT_STREQ(names[i], rec.u.ref.refname);\n \n \t\twant.len--;\n \t\tn = block_reader_seek(&br, &it, &want);\n@@ -100,7 +100,7 @@ static void test_block_read_write(void)\n \n \t\tn = block_iter_next(&it, &rec);\n \t\tEXPECT(n == 0);\n-\t\tEXPECT_STREQ(names[10 * (i / 10)], ref.refname);\n+\t\tEXPECT_STREQ(names[10 * (i / 10)], rec.u.ref.refname);\n \n \t\tblock_iter_close(&it);\n \t}\ndiff --git a/reftable/generic.c b/reftable/generic.c\nindex 7a8a738d860..b27d152e89a 100644\n--- a/reftable/generic.c\n+++ b/reftable/generic.c\n@@ -7,6 +7,7 @@ https://developers.google.com/open-source/licenses/bsd\n */\n \n #include \"basics.h\"\n+#include \"constants.h\"\n #include \"record.h\"\n #include \"generic.h\"\n #include \"reftable-iterator.h\"\n@@ -15,23 +16,21 @@ https://developers.google.com/open-source/licenses/bsd\n int reftable_table_seek_ref(struct reftable_table *tab,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_REF,\n+\t\t\t\t .u.ref = {\n+\t\t\t\t\t .refname = (char *)name,\n+\t\t\t\t } };\n \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n }\n \n int reftable_table_seek_log(struct reftable_table *tab,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = ~((uint64_t)0),\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n+\t\t\t\t .u.log = {\n+\t\t\t\t\t .refname = (char *)name,\n+\t\t\t\t\t .update_index = ~((uint64_t)0),\n+\t\t\t\t } };\n \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n }\n \n@@ -129,17 +128,25 @@ void reftable_iterator_destroy(struct reftable_iterator *it)\n int reftable_iterator_next_ref(struct reftable_iterator *it,\n \t\t\t struct reftable_ref_record *ref)\n {\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, ref);\n-\treturn iterator_next(it, &rec);\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = *ref,\n+\t};\n+\tint err = iterator_next(it, &rec);\n+\t*ref = rec.u.ref;\n+\treturn err;\n }\n \n int reftable_iterator_next_log(struct reftable_iterator *it,\n \t\t\t struct reftable_log_record *log)\n {\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, log);\n-\treturn iterator_next(it, &rec);\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.u.log = *log,\n+\t};\n+\tint err = iterator_next(it, &rec);\n+\t*log = rec.u.log;\n+\treturn err;\n }\n \n int iterator_next(struct reftable_iterator *it, struct reftable_record *rec)\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex 93d04f735b8..a8d174c0406 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -32,7 +32,7 @@ static int filtering_ref_iterator_next(void *iter_arg,\n \t\t\t\t struct reftable_record *rec)\n {\n \tstruct filtering_ref_iterator *fri = iter_arg;\n-\tstruct reftable_ref_record *ref = rec->data;\n+\tstruct reftable_ref_record *ref = &rec->u.ref;\n \tint err = 0;\n \twhile (1) {\n \t\terr = reftable_iterator_next_ref(&fri->it, ref);\n@@ -127,7 +127,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n static int indexed_table_ref_iter_next(void *p, struct reftable_record *rec)\n {\n \tstruct indexed_table_ref_iter *it = p;\n-\tstruct reftable_ref_record *ref = rec->data;\n+\tstruct reftable_ref_record *ref = &rec->u.ref;\n \n \twhile (1) {\n \t\tint err = block_iter_next(&it->cur, rec);\ndiff --git a/reftable/merged.c b/reftable/merged.c\nindex e5b53da6db3..2a6efa110d5 100644\n--- a/reftable/merged.c\n+++ b/reftable/merged.c\n@@ -30,7 +30,7 @@ static int merged_iter_init(struct merged_iter *mi)\n \n \t\tif (err > 0) {\n \t\t\treftable_iterator_destroy(&mi->stack[i]);\n-\t\t\treftable_record_destroy(&rec);\n+\t\t\treftable_record_release(&rec);\n \t\t} else {\n \t\t\tstruct pq_entry e = {\n \t\t\t\t.rec = rec,\n@@ -57,18 +57,17 @@ static void merged_iter_close(void *p)\n static int merged_iter_advance_nonnull_subiter(struct merged_iter *mi,\n \t\t\t\t\t size_t idx)\n {\n-\tstruct reftable_record rec = reftable_new_record(mi->typ);\n \tstruct pq_entry e = {\n-\t\t.rec = rec,\n+\t\t.rec = reftable_new_record(mi->typ),\n \t\t.index = idx,\n \t};\n-\tint err = iterator_next(&mi->stack[idx], &rec);\n+\tint err = iterator_next(&mi->stack[idx], &e.rec);\n \tif (err < 0)\n \t\treturn err;\n \n \tif (err > 0) {\n \t\treftable_iterator_destroy(&mi->stack[idx]);\n-\t\treftable_record_destroy(&rec);\n+\t\treftable_record_release(&e.rec);\n \t\treturn 0;\n \t}\n \n@@ -126,11 +125,11 @@ static int merged_iter_next_entry(struct merged_iter *mi,\n \t\tif (err < 0) {\n \t\t\treturn err;\n \t\t}\n-\t\treftable_record_destroy(&top.rec);\n+\t\treftable_record_release(&top.rec);\n \t}\n \n \treftable_record_copy_from(rec, &entry.rec, hash_size(mi->hash_id));\n-\treftable_record_destroy(&entry.rec);\n+\treftable_record_release(&entry.rec);\n \tstrbuf_release(&entry_key);\n \treturn 0;\n }\n@@ -290,11 +289,12 @@ int reftable_merged_table_seek_ref(struct reftable_merged_table *mt,\n \t\t\t\t struct reftable_iterator *it,\n \t\t\t\t const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = {\n+\t\t\t.refname = (char *)name,\n+\t\t},\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n \treturn merged_table_seek_record(mt, it, &rec);\n }\n \n@@ -302,12 +302,11 @@ int reftable_merged_table_seek_log_at(struct reftable_merged_table *mt,\n \t\t\t\t struct reftable_iterator *it,\n \t\t\t\t const char *name, uint64_t update_index)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = update_index,\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n+\t\t\t\t .u.log = {\n+\t\t\t\t\t .refname = (char *)name,\n+\t\t\t\t\t .update_index = update_index,\n+\t\t\t\t } };\n \treturn merged_table_seek_record(mt, it, &rec);\n }\n \ndiff --git a/reftable/pq.c b/reftable/pq.c\nindex efc474017a2..96ca6dd37b3 100644\n--- a/reftable/pq.c\n+++ b/reftable/pq.c\n@@ -74,6 +74,7 @@ struct pq_entry merged_iter_pqueue_remove(struct merged_iter_pqueue *pq)\n void merged_iter_pqueue_add(struct merged_iter_pqueue *pq, struct pq_entry e)\n {\n \tint i = 0;\n+\n \tif (pq->len == pq->cap) {\n \t\tpq->cap = 2 * pq->cap + 1;\n \t\tpq->heap = reftable_realloc(pq->heap,\n@@ -98,7 +99,7 @@ void merged_iter_pqueue_release(struct merged_iter_pqueue *pq)\n {\n \tint i = 0;\n \tfor (i = 0; i < pq->len; i++) {\n-\t\treftable_record_destroy(&pq->heap[i].rec);\n+\t\treftable_record_release(&pq->heap[i].rec);\n \t}\n \tFREE_AND_NULL(pq->heap);\n \tpq->len = pq->cap = 0;\ndiff --git a/reftable/pq_test.c b/reftable/pq_test.c\nindex c9bb05e37b7..7de5e886f35 100644\n--- a/reftable/pq_test.c\n+++ b/reftable/pq_test.c\n@@ -31,7 +31,7 @@ static void test_pq(void)\n \tint N = ARRAY_SIZE(names) - 1;\n \n \tstruct merged_iter_pqueue pq = { NULL };\n-\tconst char *last = NULL;\n+\tchar *last = NULL;\n \n \tint i = 0;\n \tfor (i = 0; i < N; i++) {\n@@ -42,12 +42,10 @@ static void test_pq(void)\n \n \ti = 1;\n \tdo {\n-\t\tstruct reftable_record rec =\n-\t\t\treftable_new_record(BLOCK_TYPE_REF);\n-\t\tstruct pq_entry e = { 0 };\n-\n-\t\treftable_record_as_ref(&rec)->refname = names[i];\n-\t\te.rec = rec;\n+\t\tstruct pq_entry e = { .rec = { .type = BLOCK_TYPE_REF,\n+\t\t\t\t\t .u.ref = {\n+\t\t\t\t\t\t .refname = names[i],\n+\t\t\t\t\t } } };\n \t\tmerged_iter_pqueue_add(&pq, e);\n \t\tmerged_iter_pqueue_check(pq);\n \t\ti = (i * 7) % N;\n@@ -55,19 +53,18 @@ static void test_pq(void)\n \n \twhile (!merged_iter_pqueue_is_empty(pq)) {\n \t\tstruct pq_entry e = merged_iter_pqueue_remove(&pq);\n-\t\tstruct reftable_ref_record *ref =\n-\t\t\treftable_record_as_ref(&e.rec);\n-\n+\t\tstruct reftable_record *rec = &e.rec;\n \t\tmerged_iter_pqueue_check(pq);\n \n+\t\tEXPECT(reftable_record_type(rec) == BLOCK_TYPE_REF);\n \t\tif (last) {\n-\t\t\tEXPECT(strcmp(last, ref->refname) < 0);\n+\t\t\tEXPECT(strcmp(last, rec->u.ref.refname) < 0);\n \t\t}\n-\t\tlast = ref->refname;\n-\t\tref->refname = NULL;\n-\t\treftable_free(ref);\n+\t\t// this is names[i], so don't dealloc.\n+\t\tlast = rec->u.ref.refname;\n+\t\trec->u.ref.refname = NULL;\n+\t\treftable_record_release(rec);\n \t}\n-\n \tfor (i = 0; i < N; i++) {\n \t\treftable_free(names[i]);\n \t}\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 272378ed1d5..00906e7a2de 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -239,8 +239,7 @@ static int table_iter_next_in_block(struct table_iter *ti,\n {\n \tint res = block_iter_next(&ti->bi, rec);\n \tif (res == 0 && reftable_record_type(rec) == BLOCK_TYPE_REF) {\n-\t\t((struct reftable_ref_record *)rec->data)->update_index +=\n-\t\t\tti->r->min_update_index;\n+\t\trec->u.ref.update_index += ti->r->min_update_index;\n \t}\n \n \treturn res;\n@@ -480,7 +479,7 @@ static int reader_seek_linear(struct reftable_reader *r, struct table_iter *ti,\n \n done:\n \tblock_iter_close(&next.bi);\n-\treftable_record_destroy(&rec);\n+\treftable_record_release(&rec);\n \tstrbuf_release(&want_key);\n \tstrbuf_release(&got_key);\n \treturn err;\n@@ -490,34 +489,35 @@ static int reader_seek_indexed(struct reftable_reader *r,\n \t\t\t struct reftable_iterator *it,\n \t\t\t struct reftable_record *rec)\n {\n-\tstruct reftable_index_record want_index = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record want_index_rec = { NULL };\n-\tstruct reftable_index_record index_result = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record index_result_rec = { NULL };\n+\tstruct reftable_record want_index = {\n+\t\t.type = BLOCK_TYPE_INDEX, .u.idx = { .last_key = STRBUF_INIT }\n+\t};\n+\tstruct reftable_record index_result = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = { .last_key = STRBUF_INIT },\n+\t};\n \tstruct table_iter index_iter = TABLE_ITER_INIT;\n \tstruct table_iter next = TABLE_ITER_INIT;\n \tint err = 0;\n \n-\treftable_record_key(rec, &want_index.last_key);\n-\treftable_record_from_index(&want_index_rec, &want_index);\n-\treftable_record_from_index(&index_result_rec, &index_result);\n-\n+\treftable_record_key(rec, &want_index.u.idx.last_key);\n \terr = reader_start(r, &index_iter, reftable_record_type(rec), 1);\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = reader_seek_linear(r, &index_iter, &want_index_rec);\n+\terr = reader_seek_linear(r, &index_iter, &want_index);\n \twhile (1) {\n-\t\terr = table_iter_next(&index_iter, &index_result_rec);\n+\t\terr = table_iter_next(&index_iter, &index_result);\n \t\ttable_iter_block_done(&index_iter);\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = reader_table_iter_at(r, &next, index_result.offset, 0);\n+\t\terr = reader_table_iter_at(r, &next, index_result.u.idx.offset,\n+\t\t\t\t\t 0);\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = block_iter_seek(&next.bi, &want_index.last_key);\n+\t\terr = block_iter_seek(&next.bi, &want_index.u.idx.last_key);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n@@ -545,8 +545,8 @@ static int reader_seek_indexed(struct reftable_reader *r,\n done:\n \tblock_iter_close(&next.bi);\n \ttable_iter_close(&index_iter);\n-\treftable_record_release(&want_index_rec);\n-\treftable_record_release(&index_result_rec);\n+\treftable_record_release(&want_index);\n+\treftable_record_release(&index_result);\n \treturn err;\n }\n \n@@ -595,11 +595,12 @@ static int reader_seek(struct reftable_reader *r, struct reftable_iterator *it,\n int reftable_reader_seek_ref(struct reftable_reader *r,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = {\n+\t\t\t.refname = (char *)name,\n+\t\t},\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n \treturn reader_seek(r, it, &rec);\n }\n \n@@ -607,12 +608,11 @@ int reftable_reader_seek_log_at(struct reftable_reader *r,\n \t\t\t\tstruct reftable_iterator *it, const char *name,\n \t\t\t\tuint64_t update_index)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = update_index,\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n+\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n+\t\t\t\t .u.log = {\n+\t\t\t\t\t .refname = (char *)name,\n+\t\t\t\t\t .update_index = update_index,\n+\t\t\t\t } };\n \treturn reader_seek(r, it, &rec);\n }\n \n@@ -656,31 +656,33 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \t\t\t\t\t struct reftable_iterator *it,\n \t\t\t\t\t uint8_t *oid)\n {\n-\tstruct reftable_obj_record want = {\n-\t\t.hash_prefix = oid,\n-\t\t.hash_prefix_len = r->object_id_len,\n+\tstruct reftable_record want = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.u.obj = {\n+\t\t\t.hash_prefix = oid,\n+\t\t\t.hash_prefix_len = r->object_id_len,\n+\t\t},\n \t};\n-\tstruct reftable_record want_rec = { NULL };\n \tstruct reftable_iterator oit = { NULL };\n-\tstruct reftable_obj_record got = { NULL };\n-\tstruct reftable_record got_rec = { NULL };\n+\tstruct reftable_record got = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.u.obj = { 0 },\n+\t};\n \tint err = 0;\n \tstruct indexed_table_ref_iter *itr = NULL;\n \n \t/* Look through the reverse index. */\n-\treftable_record_from_obj(&want_rec, &want);\n-\terr = reader_seek(r, &oit, &want_rec);\n+\terr = reader_seek(r, &oit, &want);\n \tif (err != 0)\n \t\tgoto done;\n \n \t/* read out the reftable_obj_record */\n-\treftable_record_from_obj(&got_rec, &got);\n-\terr = iterator_next(&oit, &got_rec);\n+\terr = iterator_next(&oit, &got);\n \tif (err < 0)\n \t\tgoto done;\n \n-\tif (err > 0 ||\n-\t memcmp(want.hash_prefix, got.hash_prefix, r->object_id_len)) {\n+\tif (err > 0 || memcmp(want.u.obj.hash_prefix, got.u.obj.hash_prefix,\n+\t\t\t r->object_id_len)) {\n \t\t/* didn't find it; return empty iterator */\n \t\titerator_set_empty(it);\n \t\terr = 0;\n@@ -688,15 +690,16 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \t}\n \n \terr = new_indexed_table_ref_iter(&itr, r, oid, hash_size(r->hash_id),\n-\t\t\t\t\t got.offsets, got.offset_len);\n+\t\t\t\t\t got.u.obj.offsets,\n+\t\t\t\t\t got.u.obj.offset_len);\n \tif (err < 0)\n \t\tgoto done;\n-\tgot.offsets = NULL;\n+\tgot.u.obj.offsets = NULL;\n \titerator_from_indexed_table_ref_iter(it, itr);\n \n done:\n \treftable_iterator_destroy(&oit);\n-\treftable_record_release(&got_rec);\n+\treftable_record_release(&got);\n \treturn err;\n }\n \ndiff --git a/reftable/record.c b/reftable/record.c\nindex 2a9e41a992e..a8cee628942 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -15,6 +15,10 @@ https://developers.google.com/open-source/licenses/bsd\n #include \"reftable-error.h\"\n #include \"basics.h\"\n \n+static struct reftable_record_vtable *\n+reftable_record_vtable(struct reftable_record *rec);\n+static void *reftable_record_data(struct reftable_record *rec);\n+\n int get_var_int(uint64_t *dest, struct string_view *in)\n {\n \tint ptr = 0;\n@@ -475,12 +479,14 @@ static void reftable_obj_record_copy_from(void *rec, const void *src_rec,\n \t\t(const struct reftable_obj_record *)src_rec;\n \n \treftable_obj_record_release(obj);\n-\t*obj = *src;\n-\tobj->hash_prefix = reftable_malloc(obj->hash_prefix_len);\n-\tmemcpy(obj->hash_prefix, src->hash_prefix, obj->hash_prefix_len);\n+\tobj->hash_prefix = reftable_malloc(src->hash_prefix_len);\n+\tobj->hash_prefix_len = src->hash_prefix_len;\n+\tif (src->hash_prefix_len)\n+\t\tmemcpy(obj->hash_prefix, src->hash_prefix, obj->hash_prefix_len);\n \n-\tobj->offsets = reftable_malloc(obj->offset_len * sizeof(uint64_t));\n-\tCOPY_ARRAY(obj->offsets, src->offsets, obj->offset_len);\n+\tobj->offsets = reftable_malloc(src->offset_len * sizeof(uint64_t));\n+\tobj->offset_len = src->offset_len;\n+\tCOPY_ARRAY(obj->offsets, src->offsets, src->offset_len);\n }\n \n static uint8_t reftable_obj_record_val_type(const void *rec)\n@@ -965,58 +971,6 @@ static struct reftable_record_vtable reftable_log_record_vtable = {\n \t.equal = &reftable_log_record_equal_void\n };\n \n-struct reftable_record reftable_new_record(uint8_t typ)\n-{\n-\tstruct reftable_record rec = { NULL };\n-\tswitch (typ) {\n-\tcase BLOCK_TYPE_REF: {\n-\t\tstruct reftable_ref_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_ref_record));\n-\t\treftable_record_from_ref(&rec, r);\n-\t\treturn rec;\n-\t}\n-\n-\tcase BLOCK_TYPE_OBJ: {\n-\t\tstruct reftable_obj_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_obj_record));\n-\t\treftable_record_from_obj(&rec, r);\n-\t\treturn rec;\n-\t}\n-\tcase BLOCK_TYPE_LOG: {\n-\t\tstruct reftable_log_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_log_record));\n-\t\treftable_record_from_log(&rec, r);\n-\t\treturn rec;\n-\t}\n-\tcase BLOCK_TYPE_INDEX: {\n-\t\tstruct reftable_index_record empty = { .last_key =\n-\t\t\t\t\t\t\t STRBUF_INIT };\n-\t\tstruct reftable_index_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_index_record));\n-\t\t*r = empty;\n-\t\treftable_record_from_index(&rec, r);\n-\t\treturn rec;\n-\t}\n-\t}\n-\tabort();\n-\treturn rec;\n-}\n-\n-/* clear out the record, yielding the reftable_record data that was\n- * encapsulated. */\n-static void *reftable_record_yield(struct reftable_record *rec)\n-{\n-\tvoid *p = rec->data;\n-\trec->data = NULL;\n-\treturn p;\n-}\n-\n-void reftable_record_destroy(struct reftable_record *rec)\n-{\n-\treftable_record_release(rec);\n-\treftable_free(reftable_record_yield(rec));\n-}\n-\n static void reftable_index_record_key(const void *r, struct strbuf *dest)\n {\n \tconst struct reftable_index_record *rec = r;\n@@ -1103,98 +1057,60 @@ static struct reftable_record_vtable reftable_index_record_vtable = {\n \n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)\n {\n-\trec->ops->key(rec->data, dest);\n+\treftable_record_vtable(rec)->key(reftable_record_data(rec), dest);\n }\n \n uint8_t reftable_record_type(struct reftable_record *rec)\n {\n-\treturn rec->ops->type;\n+\treturn rec->type;\n }\n \n int reftable_record_encode(struct reftable_record *rec, struct string_view dest,\n \t\t\t int hash_size)\n {\n-\treturn rec->ops->encode(rec->data, dest, hash_size);\n+\treturn reftable_record_vtable(rec)->encode(reftable_record_data(rec),\n+\t\t\t\t\t\t dest, hash_size);\n }\n \n void reftable_record_copy_from(struct reftable_record *rec,\n \t\t\t struct reftable_record *src, int hash_size)\n {\n-\tassert(src->ops->type == rec->ops->type);\n+\tassert(src->type == rec->type);\n \n-\trec->ops->copy_from(rec->data, src->data, hash_size);\n+\treftable_record_vtable(rec)->copy_from(reftable_record_data(rec),\n+\t\t\t\t\t reftable_record_data(src),\n+\t\t\t\t\t hash_size);\n }\n \n uint8_t reftable_record_val_type(struct reftable_record *rec)\n {\n-\treturn rec->ops->val_type(rec->data);\n+\treturn reftable_record_vtable(rec)->val_type(reftable_record_data(rec));\n }\n \n int reftable_record_decode(struct reftable_record *rec, struct strbuf key,\n \t\t\t uint8_t extra, struct string_view src, int hash_size)\n {\n-\treturn rec->ops->decode(rec->data, key, extra, src, hash_size);\n+\treturn reftable_record_vtable(rec)->decode(reftable_record_data(rec),\n+\t\t\t\t\t\t key, extra, src, hash_size);\n }\n \n void reftable_record_release(struct reftable_record *rec)\n {\n-\trec->ops->release(rec->data);\n+\treftable_record_vtable(rec)->release(reftable_record_data(rec));\n }\n \n int reftable_record_is_deletion(struct reftable_record *rec)\n {\n-\treturn rec->ops->is_deletion(rec->data);\n+\treturn reftable_record_vtable(rec)->is_deletion(\n+\t\treftable_record_data(rec));\n }\n \n int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size)\n {\n-\tif (a->ops != b->ops)\n+\tif (a->type != b->type)\n \t\treturn 0;\n-\treturn a->ops->equal(a->data, b->data, hash_size);\n-}\n-\n-void reftable_record_from_ref(struct reftable_record *rec,\n-\t\t\t struct reftable_ref_record *ref_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = ref_rec;\n-\trec->ops = &reftable_ref_record_vtable;\n-}\n-\n-void reftable_record_from_obj(struct reftable_record *rec,\n-\t\t\t struct reftable_obj_record *obj_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = obj_rec;\n-\trec->ops = &reftable_obj_record_vtable;\n-}\n-\n-void reftable_record_from_index(struct reftable_record *rec,\n-\t\t\t\tstruct reftable_index_record *index_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = index_rec;\n-\trec->ops = &reftable_index_record_vtable;\n-}\n-\n-void reftable_record_from_log(struct reftable_record *rec,\n-\t\t\t struct reftable_log_record *log_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = log_rec;\n-\trec->ops = &reftable_log_record_vtable;\n-}\n-\n-struct reftable_ref_record *reftable_record_as_ref(struct reftable_record *rec)\n-{\n-\tassert(reftable_record_type(rec) == BLOCK_TYPE_REF);\n-\treturn rec->data;\n-}\n-\n-struct reftable_log_record *reftable_record_as_log(struct reftable_record *rec)\n-{\n-\tassert(reftable_record_type(rec) == BLOCK_TYPE_LOG);\n-\treturn rec->data;\n+\treturn reftable_record_vtable(a)->equal(\n+\t\treftable_record_data(a), reftable_record_data(b), hash_size);\n }\n \n static int hash_equal(uint8_t *a, uint8_t *b, int hash_size)\n@@ -1267,3 +1183,75 @@ void string_view_consume(struct string_view *s, int n)\n \ts->buf += n;\n \ts->len -= n;\n }\n+\n+static void *reftable_record_data(struct reftable_record *rec)\n+{\n+\tswitch (rec->type) {\n+\tcase BLOCK_TYPE_REF:\n+\t\treturn &rec->u.ref;\n+\tcase BLOCK_TYPE_LOG:\n+\t\treturn &rec->u.log;\n+\tcase BLOCK_TYPE_INDEX:\n+\t\treturn &rec->u.idx;\n+\tcase BLOCK_TYPE_OBJ:\n+\t\treturn &rec->u.obj;\n+\t}\n+\tabort();\n+}\n+\n+static struct reftable_record_vtable *\n+reftable_record_vtable(struct reftable_record *rec)\n+{\n+\tswitch (rec->type) {\n+\tcase BLOCK_TYPE_REF:\n+\t\treturn &reftable_ref_record_vtable;\n+\tcase BLOCK_TYPE_LOG:\n+\t\treturn &reftable_log_record_vtable;\n+\tcase BLOCK_TYPE_INDEX:\n+\t\treturn &reftable_index_record_vtable;\n+\tcase BLOCK_TYPE_OBJ:\n+\t\treturn &reftable_obj_record_vtable;\n+\t}\n+\tabort();\n+}\n+\n+struct reftable_record reftable_new_record(uint8_t typ)\n+{\n+\tstruct reftable_record clean = {\n+\t\t.type = typ,\n+\t};\n+\n+\t/* the following is involved, but the naive solution (just return\n+\t * `clean` as is, except for BLOCK_TYPE_INDEX), returns a garbage\n+\t * clean.u.obj.offsets pointer on Windows VS CI. Go figure.\n+\t */\n+\tswitch (typ) {\n+\tcase BLOCK_TYPE_OBJ:\n+\t{\n+\t\tstruct reftable_obj_record obj = { 0 };\n+\t\tclean.u.obj = obj;\n+\t\tbreak;\n+\t}\n+\tcase BLOCK_TYPE_INDEX:\n+\t{\n+\t\tstruct reftable_index_record idx = {\n+\t\t\t.last_key = STRBUF_INIT,\n+\t\t};\n+\t\tclean.u.idx = idx;\n+\t\tbreak;\n+\t}\n+\tcase BLOCK_TYPE_REF:\n+\t{\n+\t\tstruct reftable_ref_record ref = { 0 };\n+\t\tclean.u.ref = ref;\n+\t\tbreak;\n+\t}\n+\tcase BLOCK_TYPE_LOG:\n+\t{\n+\t\tstruct reftable_log_record log = { 0 };\n+\t\tclean.u.log = log;\n+\t\tbreak;\n+\t}\n+\t}\n+\treturn clean;\n+}\ndiff --git a/reftable/record.h b/reftable/record.h\nindex da75d7d1f11..010a322e901 100644\n--- a/reftable/record.h\n+++ b/reftable/record.h\n@@ -63,16 +63,10 @@ struct reftable_record_vtable {\n \tint (*equal)(const void *a, const void *b, int hash_size);\n };\n \n-/* record is a generic wrapper for different types of records. */\n-struct reftable_record {\n-\tvoid *data;\n-\tstruct reftable_record_vtable *ops;\n-};\n-\n /* returns true for recognized block types. Block start with the block type. */\n int reftable_is_block_type(uint8_t typ);\n \n-/* creates a malloced record of the given type. Dispose with record_destroy */\n+/* return an initialized record for the given type */\n struct reftable_record reftable_new_record(uint8_t typ);\n \n /* Encode `key` into `dest`. Sets `is_restart` to indicate a restart. Returns\n@@ -100,6 +94,22 @@ struct reftable_obj_record {\n \tint offset_len;\n };\n \n+/* record is a generic wrapper for different types of records. It is normally\n+ * created on the stack, or embedded within another struct. If the type is\n+ * known, a fresh instance can be initialized explicitly. Otherwise, use\n+ * reftable_new_record() to initialize generically (as the index_record is not\n+ * valid as 0-initialized structure)\n+ */\n+struct reftable_record {\n+\tuint8_t type;\n+\tunion {\n+\t\tstruct reftable_ref_record ref;\n+\t\tstruct reftable_log_record log;\n+\t\tstruct reftable_obj_record obj;\n+\t\tstruct reftable_index_record idx;\n+\t} u;\n+};\n+\n /* see struct record_vtable */\n int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size);\n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest);\n@@ -114,25 +124,9 @@ int reftable_record_decode(struct reftable_record *rec, struct strbuf key,\n \t\t\t int hash_size);\n int reftable_record_is_deletion(struct reftable_record *rec);\n \n-/* zeroes out the embedded record */\n+/* frees and zeroes out the embedded record */\n void reftable_record_release(struct reftable_record *rec);\n \n-/* clear and deallocate embedded record, and zero `rec`. */\n-void reftable_record_destroy(struct reftable_record *rec);\n-\n-/* initialize generic records from concrete records. The generic record should\n- * be zeroed out. */\n-void reftable_record_from_obj(struct reftable_record *rec,\n-\t\t\t struct reftable_obj_record *objrec);\n-void reftable_record_from_index(struct reftable_record *rec,\n-\t\t\t\tstruct reftable_index_record *idxrec);\n-void reftable_record_from_ref(struct reftable_record *rec,\n-\t\t\t struct reftable_ref_record *refrec);\n-void reftable_record_from_log(struct reftable_record *rec,\n-\t\t\t struct reftable_log_record *logrec);\n-struct reftable_ref_record *reftable_record_as_ref(struct reftable_record *ref);\n-struct reftable_log_record *reftable_record_as_log(struct reftable_record *ref);\n-\n /* for qsort. */\n int reftable_ref_record_compare_name(const void *a, const void *b);\n \ndiff --git a/reftable/record_test.c b/reftable/record_test.c\nindex 92680848156..c6fdd1925a9 100644\n--- a/reftable/record_test.c\n+++ b/reftable/record_test.c\n@@ -16,13 +16,16 @@\n \n static void test_copy(struct reftable_record *rec)\n {\n-\tstruct reftable_record copy =\n-\t\treftable_new_record(reftable_record_type(rec));\n+\tstruct reftable_record copy = { 0 };\n+\tuint8_t typ;\n+\n+\ttyp = reftable_record_type(rec);\n+\tcopy = reftable_new_record(typ);\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \t/* do it twice to catch memory leaks */\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \tEXPECT(reftable_record_equal(rec, ©, GIT_SHA1_RAWSZ));\n-\treftable_record_destroy(©);\n+\treftable_record_release(©);\n }\n \n static void test_varint_roundtrip(void)\n@@ -95,61 +98,58 @@ static void test_reftable_ref_record_roundtrip(void)\n \tint i = 0;\n \n \tfor (i = REFTABLE_REF_DELETION; i < REFTABLE_NR_REF_VALUETYPES; i++) {\n-\t\tstruct reftable_ref_record in = { NULL };\n-\t\tstruct reftable_ref_record out = { NULL };\n-\t\tstruct reftable_record rec_out = { NULL };\n+\t\tstruct reftable_record in = {\n+\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t};\n+\t\tstruct reftable_record out = { .type = BLOCK_TYPE_REF };\n \t\tstruct strbuf key = STRBUF_INIT;\n-\t\tstruct reftable_record rec = { NULL };\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n \t\t\t.buf = buffer,\n \t\t\t.len = sizeof(buffer),\n \t\t};\n-\n \t\tint n, m;\n \n-\t\tin.value_type = i;\n+\t\tin.u.ref.value_type = i;\n \t\tswitch (i) {\n \t\tcase REFTABLE_REF_DELETION:\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_VAL1:\n-\t\t\tin.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val1, 1);\n+\t\t\tin.u.ref.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n+\t\t\tset_hash(in.u.ref.value.val1, 1);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_VAL2:\n-\t\t\tin.value.val2.value = reftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val2.value, 1);\n-\t\t\tin.value.val2.target_value =\n+\t\t\tin.u.ref.value.val2.value =\n \t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val2.target_value, 2);\n+\t\t\tset_hash(in.u.ref.value.val2.value, 1);\n+\t\t\tin.u.ref.value.val2.target_value =\n+\t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n+\t\t\tset_hash(in.u.ref.value.val2.target_value, 2);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_SYMREF:\n-\t\t\tin.value.symref = xstrdup(\"target\");\n+\t\t\tin.u.ref.value.symref = xstrdup(\"target\");\n \t\t\tbreak;\n \t\t}\n-\t\tin.refname = xstrdup(\"refs/heads/master\");\n+\t\tin.u.ref.refname = xstrdup(\"refs/heads/master\");\n \n-\t\treftable_record_from_ref(&rec, &in);\n-\t\ttest_copy(&rec);\n+\t\ttest_copy(&in);\n \n-\t\tEXPECT(reftable_record_val_type(&rec) == i);\n+\t\tEXPECT(reftable_record_val_type(&in) == i);\n \n-\t\treftable_record_key(&rec, &key);\n-\t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\t\treftable_record_key(&in, &key);\n+\t\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n > 0);\n \n \t\t/* decode into a non-zero reftable_record to test for leaks. */\n-\n-\t\treftable_record_from_ref(&rec_out, &out);\n-\t\tm = reftable_record_decode(&rec_out, key, i, dest,\n-\t\t\t\t\t GIT_SHA1_RAWSZ);\n+\t\tm = reftable_record_decode(&out, key, i, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_ref_record_equal(&in, &out, GIT_SHA1_RAWSZ));\n-\t\treftable_record_release(&rec_out);\n+\t\tEXPECT(reftable_ref_record_equal(&in.u.ref, &out.u.ref,\n+\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n+\t\treftable_record_release(&in);\n \n \t\tstrbuf_release(&key);\n-\t\treftable_ref_record_release(&in);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n@@ -202,7 +202,7 @@ static void test_reftable_log_record_roundtrip(void)\n \tset_test_hash(in[0].value.update.new_hash, 1);\n \tset_test_hash(in[0].value.update.old_hash, 2);\n \tfor (i = 0; i < ARRAY_SIZE(in); i++) {\n-\t\tstruct reftable_record rec = { NULL };\n+\t\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG };\n \t\tstruct strbuf key = STRBUF_INIT;\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n@@ -210,23 +210,25 @@ static void test_reftable_log_record_roundtrip(void)\n \t\t\t.len = sizeof(buffer),\n \t\t};\n \t\t/* populate out, to check for leaks. */\n-\t\tstruct reftable_log_record out = {\n-\t\t\t.refname = xstrdup(\"old name\"),\n-\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n-\t\t\t.value = {\n-\t\t\t\t.update = {\n-\t\t\t\t\t.new_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n-\t\t\t\t\t.old_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n-\t\t\t\t\t.name = xstrdup(\"old name\"),\n-\t\t\t\t\t.email = xstrdup(\"old@email\"),\n-\t\t\t\t\t.message = xstrdup(\"old message\"),\n+\t\tstruct reftable_record out = {\n+\t\t\t.type = BLOCK_TYPE_LOG,\n+\t\t\t.u.log = {\n+\t\t\t\t.refname = xstrdup(\"old name\"),\n+\t\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n+\t\t\t\t.value = {\n+\t\t\t\t\t.update = {\n+\t\t\t\t\t\t.new_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t\t.old_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t\t.name = xstrdup(\"old name\"),\n+\t\t\t\t\t\t.email = xstrdup(\"old@email\"),\n+\t\t\t\t\t\t.message = xstrdup(\"old message\"),\n+\t\t\t\t\t},\n \t\t\t\t},\n \t\t\t},\n \t\t};\n-\t\tstruct reftable_record rec_out = { NULL };\n \t\tint n, m, valtype;\n \n-\t\treftable_record_from_log(&rec, &in[i]);\n+\t\trec.u.log = in[i];\n \n \t\ttest_copy(&rec);\n \n@@ -234,16 +236,16 @@ static void test_reftable_log_record_roundtrip(void)\n \n \t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n >= 0);\n-\t\treftable_record_from_log(&rec_out, &out);\n \t\tvaltype = reftable_record_val_type(&rec);\n-\t\tm = reftable_record_decode(&rec_out, key, valtype, dest,\n+\t\tm = reftable_record_decode(&out, key, valtype, dest,\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_log_record_equal(&in[i], &out, GIT_SHA1_RAWSZ));\n+\t\tEXPECT(reftable_log_record_equal(&in[i], &out.u.log,\n+\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\treftable_log_record_release(&in[i]);\n \t\tstrbuf_release(&key);\n-\t\treftable_record_release(&rec_out);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n@@ -311,41 +313,43 @@ static void test_reftable_obj_record_roundtrip(void)\n \t\t\t\t\t } };\n \tint i = 0;\n \tfor (i = 0; i < ARRAY_SIZE(recs); i++) {\n-\t\tstruct reftable_obj_record in = recs[i];\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n \t\t\t.buf = buffer,\n \t\t\t.len = sizeof(buffer),\n \t\t};\n-\t\tstruct reftable_record rec = { NULL };\n+\t\tstruct reftable_record in = {\n+\t\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t\t.u.obj = recs[i],\n+\t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n-\t\tstruct reftable_obj_record out = { NULL };\n-\t\tstruct reftable_record rec_out = { NULL };\n+\t\tstruct reftable_record out = { .type = BLOCK_TYPE_OBJ };\n \t\tint n, m;\n \t\tuint8_t extra;\n \n-\t\treftable_record_from_obj(&rec, &in);\n-\t\ttest_copy(&rec);\n-\t\treftable_record_key(&rec, &key);\n-\t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\t\ttest_copy(&in);\n+\t\treftable_record_key(&in, &key);\n+\t\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n > 0);\n-\t\textra = reftable_record_val_type(&rec);\n-\t\treftable_record_from_obj(&rec_out, &out);\n-\t\tm = reftable_record_decode(&rec_out, key, extra, dest,\n+\t\textra = reftable_record_val_type(&in);\n+\t\tm = reftable_record_decode(&out, key, extra, dest,\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_record_equal(&rec, &rec_out, GIT_SHA1_RAWSZ));\n+\t\tEXPECT(reftable_record_equal(&in, &out, GIT_SHA1_RAWSZ));\n \t\tstrbuf_release(&key);\n-\t\treftable_record_release(&rec_out);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n static void test_reftable_index_record_roundtrip(void)\n {\n-\tstruct reftable_index_record in = {\n-\t\t.offset = 42,\n-\t\t.last_key = STRBUF_INIT,\n+\tstruct reftable_record in = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = {\n+\t\t\t.offset = 42,\n+\t\t\t.last_key = STRBUF_INIT,\n+\t\t},\n \t};\n \tuint8_t buffer[1024] = { 0 };\n \tstruct string_view dest = {\n@@ -353,31 +357,30 @@ static void test_reftable_index_record_roundtrip(void)\n \t\t.len = sizeof(buffer),\n \t};\n \tstruct strbuf key = STRBUF_INIT;\n-\tstruct reftable_record rec = { NULL };\n-\tstruct reftable_index_record out = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record out_rec = { NULL };\n+\tstruct reftable_record out = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = { .last_key = STRBUF_INIT },\n+\t};\n \tint n, m;\n \tuint8_t extra;\n \n-\tstrbuf_addstr(&in.last_key, \"refs/heads/master\");\n-\treftable_record_from_index(&rec, &in);\n-\treftable_record_key(&rec, &key);\n-\ttest_copy(&rec);\n+\tstrbuf_addstr(&in.u.idx.last_key, \"refs/heads/master\");\n+\treftable_record_key(&in, &key);\n+\ttest_copy(&in);\n \n-\tEXPECT(0 == strbuf_cmp(&key, &in.last_key));\n-\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\tEXPECT(0 == strbuf_cmp(&key, &in.u.idx.last_key));\n+\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(n > 0);\n \n-\textra = reftable_record_val_type(&rec);\n-\treftable_record_from_index(&out_rec, &out);\n-\tm = reftable_record_decode(&out_rec, key, extra, dest, GIT_SHA1_RAWSZ);\n+\textra = reftable_record_val_type(&in);\n+\tm = reftable_record_decode(&out, key, extra, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(m == n);\n \n-\tEXPECT(reftable_record_equal(&rec, &out_rec, GIT_SHA1_RAWSZ));\n+\tEXPECT(reftable_record_equal(&in, &out, GIT_SHA1_RAWSZ));\n \n-\treftable_record_release(&out_rec);\n+\treftable_record_release(&out);\n \tstrbuf_release(&key);\n-\tstrbuf_release(&in.last_key);\n+\tstrbuf_release(&in.u.idx.last_key);\n }\n \n int record_test_main(int argc, const char *argv[])\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex 83a23daf60b..944c2329ab5 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -256,8 +256,10 @@ done:\n int reftable_writer_add_ref(struct reftable_writer *w,\n \t\t\t struct reftable_ref_record *ref)\n {\n-\tstruct reftable_record rec = { NULL };\n-\tstruct reftable_ref_record copy = *ref;\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = *ref,\n+\t};\n \tint err = 0;\n \n \tif (ref->refname == NULL)\n@@ -266,8 +268,7 @@ int reftable_writer_add_ref(struct reftable_writer *w,\n \t ref->update_index > w->max_update_index)\n \t\treturn REFTABLE_API_ERROR;\n \n-\treftable_record_from_ref(&rec, ©);\n-\tcopy.update_index -= w->min_update_index;\n+\trec.u.ref.update_index -= w->min_update_index;\n \n \terr = writer_add_record(w, &rec);\n \tif (err < 0)\n@@ -306,7 +307,10 @@ int reftable_writer_add_refs(struct reftable_writer *w,\n static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n \t\t\t\t\t struct reftable_log_record *log)\n {\n-\tstruct reftable_record rec = { NULL };\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.u.log = *log,\n+\t};\n \tif (w->block_writer &&\n \t block_writer_type(w->block_writer) == BLOCK_TYPE_REF) {\n \t\tint err = writer_finish_public_section(w);\n@@ -316,8 +320,6 @@ static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n \n \tw->next -= w->pending_padding;\n \tw->pending_padding = 0;\n-\n-\treftable_record_from_log(&rec, log);\n \treturn writer_add_record(w, &rec);\n }\n \n@@ -398,8 +400,10 @@ static int writer_finish_section(struct reftable_writer *w)\n \t\tw->index_len = 0;\n \t\tw->index_cap = 0;\n \t\tfor (i = 0; i < idx_len; i++) {\n-\t\t\tstruct reftable_record rec = { NULL };\n-\t\t\treftable_record_from_index(&rec, idx + i);\n+\t\t\tstruct reftable_record rec = {\n+\t\t\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t\t\t.u.idx = idx[i],\n+\t\t\t};\n \t\t\tif (block_writer_add(w->block_writer, &rec) == 0) {\n \t\t\t\tcontinue;\n \t\t\t}\n@@ -467,17 +471,17 @@ static void write_object_record(void *void_arg, void *key)\n {\n \tstruct write_record_arg *arg = void_arg;\n \tstruct obj_index_tree_node *entry = key;\n-\tstruct reftable_obj_record obj_rec = {\n-\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n-\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n-\t\t.offsets = entry->offsets,\n-\t\t.offset_len = entry->offset_len,\n-\t};\n-\tstruct reftable_record rec = { NULL };\n+\tstruct reftable_record\n+\t\trec = { .type = BLOCK_TYPE_OBJ,\n+\t\t\t.u.obj = {\n+\t\t\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n+\t\t\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n+\t\t\t\t.offsets = entry->offsets,\n+\t\t\t\t.offset_len = entry->offset_len,\n+\t\t\t} };\n \tif (arg->err < 0)\n \t\tgoto done;\n \n-\treftable_record_from_obj(&rec, &obj_rec);\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \tif (arg->err == 0)\n \t\tgoto done;\n@@ -490,7 +494,8 @@ static void write_object_record(void *void_arg, void *key)\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \tif (arg->err == 0)\n \t\tgoto done;\n-\tobj_rec.offset_len = 0;\n+\n+\trec.u.obj.offset_len = 0;\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \n \t/* Should be able to write into a fresh block. */\n-- \ngitgitgadget\n\n"},{"id":"446621","messageId":"220121.86a6fpl3hr.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"9218bd59b2570ab08f3a2e5c0b590656d00482c4.1642691534.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v6 01/15] reftable: fix OOB stack write in print functions","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-21T11:41:57Z","receivedAt":"2022-01-21T11:42:13Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jan 20 2022, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> Signed-off-by: Han-Wen Nienhuys \n> ---\n> reftable/record.c | 4 ++--\n> 1 file changed, 2 insertions(+), 2 deletions(-)\n>\n> diff --git a/reftable/record.c b/reftable/record.c\n> index 6a5dac32dc6..8536bd03aa9 100644\n> --- a/reftable/record.c\n> +++ b/reftable/record.c\n> @@ -254,7 +254,7 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)\n> void reftable_ref_record_print(struct reftable_ref_record *ref,\n> \t\t\t uint32_t hash_id)\n> {\n> -\tchar hex[2 * GIT_SHA256_RAWSZ + 1] = { 0 }; /* BUG */\n> +\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */\n\nIs whatever \"BUG\" this is still current after this change?\n"},{"id":"446622","messageId":"220121.865yqdl3ff.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"315ce62e710e6e80dc8a4d959e5532b5f9142669.1642691534.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v6 02/15] reftable: fix resource leak in block.c error path","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-21T11:42:21Z","receivedAt":"2022-01-21T11:43:41Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jan 20 2022, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> Add test coverage for corrupt zlib data. Fix memory leaks demonstrated by\n> unittest.\n>\n> This problem was discovered by a Coverity scan.\n>\n> Signed-off-by: Han-Wen Nienhuys \n> ---\n> reftable/block.c | 26 +++++++++------\n> reftable/reader.c | 23 ++++++++------\n> reftable/readwrite_test.c | 66 +++++++++++++++++++++++++++++++++++++++\n> 3 files changed, 97 insertions(+), 18 deletions(-)\n>\n> diff --git a/reftable/block.c b/reftable/block.c\n> index 855e3f5c947..6c8e8705205 100644\n> --- a/reftable/block.c\n> +++ b/reftable/block.c\n> @@ -188,13 +188,16 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n> \tuint32_t full_block_size = table_block_size;\n> \tuint8_t typ = block->data[header_off];\n> \tuint32_t sz = get_be24(block->data + header_off + 1);\n> -\n\nstray extra whitespace being added in the middle of variable\ndeclarations.\n"},{"id":"446623","messageId":"220121.861r11l3d2.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"8740ba92981c77f85b855e69c237fdb7449cca58.1642691534.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v6 04/15] reftable: check reftable_stack_auto_compact() return value","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-21T11:44:15Z","receivedAt":"2022-01-21T11:45:02Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jan 20 2022, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> Fixes a problem detected by Coverity.\n\nDetected by coverity, but I think it's better to describe the\nissue. Here it seems to be that we didn't use the \"err\" at all, so\nbefore this change the test wasn't doing anything to check that we\ndidn't have an error?\n\nOr would the subsequent EXPECT() catch it in practice?\n\n> Signed-off-by: Han-Wen Nienhuys \n> ---\n> reftable/stack_test.c | 1 +\n> 1 file changed, 1 insertion(+)\n>\n> diff --git a/reftable/stack_test.c b/reftable/stack_test.c\n> index f4c743db80c..e84f50d27ff 100644\n> --- a/reftable/stack_test.c\n> +++ b/reftable/stack_test.c\n> @@ -839,6 +839,7 @@ static void test_reftable_stack_auto_compaction(void)\n> \t\tEXPECT_ERR(err);\n> \n> \t\terr = reftable_stack_auto_compact(st);\n> +\t\tEXPECT_ERR(err);\n> \t\tEXPECT(i < 3 || st->merged->stack_len < 2 * fastlog2(i));\n> \t}\n\n"},{"id":"446624","messageId":"220121.86wnitjoke.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"08be6d90a4890b63fe9b0885af0df084c9ca81fd.1642691534.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v6 05/15] reftable: ignore remove() return value in stack_test.c","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-21T11:46:05Z","receivedAt":"2022-01-21T11:49:58Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jan 20 2022, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> If the cleanup fails, there is nothing we can do.\n>\n> Signed-off-by: Han-Wen Nienhuys \n> ---\n> reftable/stack_test.c | 2 +-\n> 1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/reftable/stack_test.c b/reftable/stack_test.c\n> index e84f50d27ff..19fe4e20085 100644\n> --- a/reftable/stack_test.c\n> +++ b/reftable/stack_test.c\n> @@ -90,7 +90,7 @@ static void test_read_file(void)\n> \t\tEXPECT(0 == strcmp(want[i], names[i]));\n> \t}\n> \tfree_names(names);\n> -\tremove(fn);\n> +\t(void) remove(fn);\n> }\n> \n> static void test_parse_names(void)\n\nWell, if we fail here due to a permission error or other I/O weirdness\nsurely it's better to:\n\n\tif (remove(fn) < 0)\n\t\tdie_errno(\"unable to remove '%s'\", fn);\n\nOtherwise we're just silently sweeping that under the rug, and likely\nhaving the \"rm -rf\" we'll shortly do in test-lib.sh catch it at a\ndistance.\n\nAlso why are we using remove() here at all? Shouldn't this just be\nunlink()? Or per the feedback above unlink_or_warn() or\nremove_or_warn()? I.e. looking at the context we just open()'d this\n\"fn\", so we're not unsure if it's a directory or a file, are we?\n\n"},{"id":"446625","messageId":"220121.86sfthjocg.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"6385e449ba714e3ce41f10276fc60e9757519492.1642691534.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v6 12/15] reftable: implement record equality generically","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-21T11:52:09Z","receivedAt":"2022-01-21T11:54:44Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jan 20 2022, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> This simplifies unittests a little, and provides further coverage for\n> reftable_record_copy().\n>\n> Signed-off-by: Han-Wen Nienhuys \n> ---\n> reftable/record.c | 57 +++++++++++++++++++++++++++++++++++++++++-\n> reftable/record.h | 5 +++-\n> reftable/record_test.c | 23 +++--------------\n> 3 files changed, 63 insertions(+), 22 deletions(-)\n>\n> diff --git a/reftable/record.c b/reftable/record.c\n> index f7c77c51539..2a9e41a992e 100644\n> --- a/reftable/record.c\n> +++ b/reftable/record.c\n> @@ -430,6 +430,15 @@ static int reftable_ref_record_is_deletion_void(const void *p)\n> \t\t(const struct reftable_ref_record *)p);\n> }\n> \n> +\n\nstray extra newline being added here.\n\n> +static int reftable_ref_record_equal_void(const void *a,\n> +\t\t\t\t\t const void *b, int hash_size)\n> +{\n> +\tstruct reftable_ref_record *ra = (struct reftable_ref_record *) a;\n> +\tstruct reftable_ref_record *rb = (struct reftable_ref_record *) b;\n> +\treturn reftable_ref_record_equal(ra, rb, hash_size);\n> +}\n> +\n> static struct reftable_record_vtable reftable_ref_record_vtable = {\n> \t.key = &reftable_ref_record_key,\n> \t.type = BLOCK_TYPE_REF,\n> @@ -439,6 +448,7 @@ static struct reftable_record_vtable reftable_ref_record_vtable = {\n> \t.decode = &reftable_ref_record_decode,\n> \t.release = &reftable_ref_record_release_void,\n> \t.is_deletion = &reftable_ref_record_is_deletion_void,\n> +\t.equal = &reftable_ref_record_equal_void,\n> };\n> \n> static void reftable_obj_record_key(const void *r, struct strbuf *dest)\n> @@ -572,6 +582,25 @@ static int not_a_deletion(const void *p)\n> \treturn 0;\n> }\n> \n> +static int reftable_obj_record_equal_void(const void *a, const void *b, int hash_size)\n> +{\n> +\tstruct reftable_obj_record *ra = (struct reftable_obj_record *) a;\n> +\tstruct reftable_obj_record *rb = (struct reftable_obj_record *) b;\n> +\n> +\tif (ra->hash_prefix_len != rb->hash_prefix_len\n> +\t || ra->offset_len != rb->offset_len)\n> +\t\treturn 0;\n> +\n> +\tif (ra->hash_prefix_len &&\n> +\t memcmp(ra->hash_prefix, rb->hash_prefix, ra->hash_prefix_len))\n> +\t\treturn 0;\n\nSimilar to the memcpy() paranoia isn't this memcmp() paranoia?\nI.e. memcmp() returns 0 on a n=0, so we can lose the\n\"ra->hash_prefix_len &&\" here, no?\n\n> +\tif (ra->offset_len &&\n> +\t memcmp(ra->offsets, rb->offsets, ra->offset_len * sizeof(uint64_t)))\n> +\t\treturn 0;\n\n...and here, since 0 * sizeof() will just get us zero.\n"},{"id":"446627","messageId":"220121.86k0etjnsq.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"69c4a55488209f96ae605047ecd1709678518fa5.1642691534.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v6 13/15] reftable: remove outdated file reftable.c","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-21T12:05:43Z","receivedAt":"2022-01-21T12:06:43Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jan 20 2022, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> This was renamed to generic.c, but the origin was never removed\n\nI thought \"hrm, didn't I have a way to assert this?\" before I remembered\nthat yes, we could use $(wildcard) in the Makefile to avoid this class\nof problem in the future:\nhttps://lore.kernel.org/git/220121.86o845jnvv.gmgdl@evledraar.gmail.com/\n\nJust food for thought :) The change itself LGTM.\n"},{"id":"446628","messageId":"220121.86fsphjml0.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"54bcac325bd542aa81328e5616b2b7573d43adac.1642691534.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v6 14/15] reftable: make reftable_record a tagged union","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-21T12:06:51Z","receivedAt":"2022-01-21T12:32:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jan 20 2022, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> This reduces the amount of glue code, because we don't need a void\n> pointer or vtable within the structure.\n> [...]\n> -\tstruct reftable_record rec = { NULL };\n> -\treftable_record_from_ref(&rec, &ref);\n> +\tstruct reftable_record rec = { .type = BLOCK_TYPE_REF,\n> +\t\t\t\t .u.ref = {\n> +\t\t\t\t\t .refname = (char *)name,\n> +\t\t\t\t } };\n> \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n> }\n\nBoth for this & the below don't we prefer to have such assignments on\ntheir own line? I.e.:\n\t\n\tstruct reftable_record rec = {\n\t\t.type = BLOCK_TYPE_REF,\n\t .u.ref = { .refname = (char *)name },\n\t};\n\nIn this case the snuggling is buying is nothing in terms of line count,\nand for the .u.ref assignment we were split across 3 lines instead of\n1. So this seems easier to read to me....\n\t\n> int reftable_table_seek_log(struct reftable_table *tab,\n> \t\t\t struct reftable_iterator *it, const char *name)\n> {\n> -\tstruct reftable_log_record log = {\n> -\t\t.refname = (char *)name,\n> -\t\t.update_index = ~((uint64_t)0),\n> -\t};\n> -\tstruct reftable_record rec = { NULL };\n> -\treftable_record_from_log(&rec, &log);\n> +\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n> +\t\t\t\t .u.log = {\n> +\t\t\t\t\t .refname = (char *)name,\n> +\t\t\t\t\t .update_index = ~((uint64_t)0),\n> +\t\t\t\t } };\n\n...ditto..\n\n> \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n> }\n> \n> @@ -129,17 +128,25 @@ void reftable_iterator_destroy(struct reftable_iterator *it)\n> int reftable_iterator_next_ref(struct reftable_iterator *it,\n> \t\t\t struct reftable_ref_record *ref)\n> {\n> -\tstruct reftable_record rec = { NULL };\n> -\treftable_record_from_ref(&rec, ref);\n> -\treturn iterator_next(it, &rec);\n> +\tstruct reftable_record rec = {\n> +\t\t.type = BLOCK_TYPE_REF,\n> +\t\t.u.ref = *ref,\n> +\t};\n\n...and here we have the sort of assignment I'm suggesting, not\n\n\tstruct reftable_record rec = { .type = [...]\n\n...\n\n> int reftable_iterator_next_log(struct reftable_iterator *it,\n> \t\t\t struct reftable_log_record *log)\n> {\n> -\tstruct reftable_record rec = { NULL };\n> -\treftable_record_from_log(&rec, log);\n> -\treturn iterator_next(it, &rec);\n> +\tstruct reftable_record rec = {\n> +\t\t.type = BLOCK_TYPE_LOG,\n> +\t\t.u.log = *log,\n> +\t};\n\n...ditto (good)....\n\n> -\tstruct reftable_ref_record ref = {\n> -\t\t.refname = (char *)name,\n> +\tstruct reftable_record rec = {\n> +\t\t.type = BLOCK_TYPE_REF,\n> +\t\t.u.ref = {\n> +\t\t\t.refname = (char *)name,\n> +\t\t},\n\n... yet another style that's a mix between the two. I suppose always\nputting the braces on their own line is consistent, but for these\none-member assignments just doing:\n\n\tstruct reftable_record rec = {\n\t\t.type = BLOCK_TYPE_REF,\n\t\t.u.ref = { .refname = (char *)name },\n \t};\n\nseems easier to read....\n\n> +\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n> +\t\t\t\t .u.log = {\n> +\t\t\t\t\t .refname = (char *)name,\n> +\t\t\t\t\t .update_index = update_index,\n> +\t\t\t\t } };\n\n...ditto (exhaustively wrapped)\n\n> \treturn merged_table_seek_record(mt, it, &rec);\n> }\n> \n> diff --git a/reftable/pq.c b/reftable/pq.c\n> index efc474017a2..96ca6dd37b3 100644\n> --- a/reftable/pq.c\n> +++ b/reftable/pq.c\n> @@ -74,6 +74,7 @@ struct pq_entry merged_iter_pqueue_remove(struct merged_iter_pqueue *pq)\n> void merged_iter_pqueue_add(struct merged_iter_pqueue *pq, struct pq_entry e)\n> {\n> \tint i = 0;\n> +\n> \tif (pq->len == pq->cap) {\n\nA stray whitespace change. A good one, but maybe (depending on others in\nthe topic) bundle those into their own patch (or maybe it's not worth\nit, haven't looked at the whole thing again).\n\nIn this case we don't otherwise touch this function, so maybe just\nleaving it for some future re-formatting change is better?\n\n> -\t\tlast = ref->refname;\n> -\t\tref->refname = NULL;\n> -\t\treftable_free(ref);\n> +\t\t// this is names[i], so don't dealloc.\n> +\t\tlast = rec->u.ref.refname;\n> +\t\trec->u.ref.refname = NULL;\n> +\t\treftable_record_release(rec);\n\nPer https://lore.kernel.org/git/87wnmwpwyf.fsf@evledraar.gmail.com/ I\nthink Junio at al prefer not to have C++-style comments in git, even\nthough they're also in C99.\n\n> \t}\n> -\n\n...more stray whitespace...\n\n> [...]\n> -\tstruct reftable_index_record want_index = { .last_key = STRBUF_INIT };\n> -\tstruct reftable_record want_index_rec = { NULL };\n> -\tstruct reftable_index_record index_result = { .last_key = STRBUF_INIT };\n> -\tstruct reftable_record index_result_rec = { NULL };\n> +\tstruct reftable_record want_index = {\n> +\t\t.type = BLOCK_TYPE_INDEX, .u.idx = { .last_key = STRBUF_INIT }\n> +\t};\n> +\tstruct reftable_record index_result = {\n> +\t\t.type = BLOCK_TYPE_INDEX,\n> +\t\t.u.idx = { .last_key = STRBUF_INIT },\n> +\t};\n\nHere you are formatting/indenting these struct assignments exactly in\nthe style I suggested above (but within just this patch I think it's the\n3rd or 4th style you're using picking).\n\n> \tstruct table_iter index_iter = TABLE_ITER_INIT;\n> \tstruct table_iter next = TABLE_ITER_INIT;\n> \tint err = 0;\n> \n> -\treftable_record_key(rec, &want_index.last_key);\n> -\treftable_record_from_index(&want_index_rec, &want_index);\n> -\treftable_record_from_index(&index_result_rec, &index_result);\n> -\n> +\treftable_record_key(rec, &want_index.u.idx.last_key);\n\nSince you're adjusting whitespace that splits blocks for readability,\ndoesn't it mke sense to retain the reftable_record_key() as a \"block\"\nhere before reader_start(), even if there's only one of them left?\n\nI.e. to split up data setup from \"real work\" in the function.\n\n> {\n> -\tstruct reftable_ref_record ref = {\n> -\t\t.refname = (char *)name,\n> +\tstruct reftable_record rec = {\n> +\t\t.type = BLOCK_TYPE_REF,\n> +\t\t.u.ref = {\n> +\t\t\t.refname = (char *)name,\n> +\t\t},\n> \t};\n> -\tstruct reftable_record rec = { NULL };\n> -\treftable_record_from_ref(&rec, &ref);\n> \treturn reader_seek(r, it, &rec);\n> }\n\n...more struct formatting...\n\n> @@ -607,12 +608,11 @@ int reftable_reader_seek_log_at(struct reftable_reader *r,\n> \t\t\t\tstruct reftable_iterator *it, const char *name,\n> \t\t\t\tuint64_t update_index)\n> {\n> -\tstruct reftable_log_record log = {\n> -\t\t.refname = (char *)name,\n> -\t\t.update_index = update_index,\n> -\t};\n> -\tstruct reftable_record rec = { NULL };\n> -\treftable_record_from_log(&rec, &log);\n> +\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n> +\t\t\t\t .u.log = {\n> +\t\t\t\t\t .refname = (char *)name,\n> +\t\t\t\t\t .update_index = update_index,\n> +\t\t\t\t } };\n\n..ditto...\n\n> \treturn reader_seek(r, it, &rec);\n> }\n> \n> @@ -656,31 +656,33 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n> \t\t\t\t\t struct reftable_iterator *it,\n> \t\t\t\t\t uint8_t *oid)\n> {\n> -\tstruct reftable_obj_record want = {\n> -\t\t.hash_prefix = oid,\n> -\t\t.hash_prefix_len = r->object_id_len,\n> +\tstruct reftable_record want = {\n> +\t\t.type = BLOCK_TYPE_OBJ,\n> +\t\t.u.obj = {\n> +\t\t\t.hash_prefix = oid,\n> +\t\t\t.hash_prefix_len = r->object_id_len,\n> +\t\t},\n> \t};\n\n..ditto...\n\n> -\tstruct reftable_record want_rec = { NULL };\n> \tstruct reftable_iterator oit = { NULL };\n> -\tstruct reftable_obj_record got = { NULL };\n> -\tstruct reftable_record got_rec = { NULL };\n> +\tstruct reftable_record got = {\n> +\t\t.type = BLOCK_TYPE_OBJ,\n> +\t\t.u.obj = { 0 },\n> +\t};\n> \tint err = 0;\n> \tstruct indexed_table_ref_iter *itr = NULL;\n> \n> \t/* Look through the reverse index. */\n> -\treftable_record_from_obj(&want_rec, &want);\n> -\terr = reader_seek(r, &oit, &want_rec);\n> +\terr = reader_seek(r, &oit, &want);\n> \tif (err != 0)\n> \t\tgoto done;\n> \n> \t/* read out the reftable_obj_record */\n> -\treftable_record_from_obj(&got_rec, &got);\n> -\terr = iterator_next(&oit, &got_rec);\n> +\terr = iterator_next(&oit, &got);\n> \tif (err < 0)\n> \t\tgoto done;\n> \n> -\tif (err > 0 ||\n> -\t memcmp(want.hash_prefix, got.hash_prefix, r->object_id_len)) {\n> +\tif (err > 0 || memcmp(want.u.obj.hash_prefix, got.u.obj.hash_prefix,\n> +\t\t\t r->object_id_len)) {\n> \t\t/* didn't find it; return empty iterator */\n> \t\titerator_set_empty(it);\n> \t\terr = 0;\n> @@ -688,15 +690,16 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n> \t}\n> \n> \terr = new_indexed_table_ref_iter(&itr, r, oid, hash_size(r->hash_id),\n> -\t\t\t\t\t got.offsets, got.offset_len);\n> +\t\t\t\t\t got.u.obj.offsets,\n> +\t\t\t\t\t got.u.obj.offset_len);\n> \tif (err < 0)\n> \t\tgoto done;\n> -\tgot.offsets = NULL;\n> +\tgot.u.obj.offsets = NULL;\n> \titerator_from_indexed_table_ref_iter(it, itr);\n> \n> done:\n> \treftable_iterator_destroy(&oit);\n> -\treftable_record_release(&got_rec);\n> +\treftable_record_release(&got);\n> \treturn err;\n> }\n> \n> diff --git a/reftable/record.c b/reftable/record.c\n> index 2a9e41a992e..a8cee628942 100644\n> --- a/reftable/record.c\n> +++ b/reftable/record.c\n> @@ -15,6 +15,10 @@ https://developers.google.com/open-source/licenses/bsd\n> #include \"reftable-error.h\"\n> #include \"basics.h\"\n> \n> +static struct reftable_record_vtable *\n> +reftable_record_vtable(struct reftable_record *rec);\n> +static void *reftable_record_data(struct reftable_record *rec);\n\nThe reftable_record_data() function can just be moved above its first\nuser, and we can avoid this forward deceleration for a new static\nfunction.\n\n> +\tif (src->hash_prefix_len)\n> +\t\tmemcpy(obj->hash_prefix, src->hash_prefix, obj->hash_prefix_len);\n\nEarlier I commented on other mem*() usage, but this goes against the\nmemcpy() paranoia being dropped referenced in the CL. I.e. we can do\naway with the guard clause here too.\n\n> +static void *reftable_record_data(struct reftable_record *rec)\n> +{\n> +\tswitch (rec->type) {\n> +\tcase BLOCK_TYPE_REF:\n> +\t\treturn &rec->u.ref;\n> +\tcase BLOCK_TYPE_LOG:\n> +\t\treturn &rec->u.log;\n> +\tcase BLOCK_TYPE_INDEX:\n> +\t\treturn &rec->u.idx;\n> +\tcase BLOCK_TYPE_OBJ:\n> +\t\treturn &rec->u.obj;\n> +\t}\n> +\tabort();\n> +}\n\nThe static function we forward-declared above, and could just move up.\n\n> +struct reftable_record reftable_new_record(uint8_t typ)\n> +{\n\nnit: There's no \"type\" variable in the below function, so just call this \"type\"?...\n\n> +\tstruct reftable_record clean = {\n> +\t\t.type = typ,\n\n...only this \"type\" member, which won't confuse a compiler.\n\n> +\tswitch (typ) {\n\n...but actually...\n\n\n> +\tcase BLOCK_TYPE_OBJ:\n> +\t{\n> +\t\tstruct reftable_obj_record obj = { 0 };\n> +\t\tclean.u.obj = obj;\n> +\t\tbreak;\n> +\t}\n> +\tcase BLOCK_TYPE_INDEX:\n> +\t{\n> +\t\tstruct reftable_index_record idx = {\n> +\t\t\t.last_key = STRBUF_INIT,\n> +\t\t};\n> +\t\tclean.u.idx = idx;\n> +\t\tbreak;\n> +\t}\n> +\tcase BLOCK_TYPE_REF:\n> +\t{\n> +\t\tstruct reftable_ref_record ref = { 0 };\n> +\t\tclean.u.ref = ref;\n> +\t\tbreak;\n> +\t}\n> +\tcase BLOCK_TYPE_LOG:\n> +\t{\n> +\t\tstruct reftable_log_record log = { 0 };\n> +\t\tclean.u.log = log;\n> +\t\tbreak;\n> +\t}\n> +\t}\n> +\treturn clean;\n> +}\n\n...reading this whole function shouldn't this be a more normal *_init()\npattern function where the caller just populates the \".type = \", and we\ninit the rest here? That would also make the ownership more obvious, and\nif any future API user needs to pass in variable on the heap instead of\nus returning it on the stack here...\n\n> -/* zeroes out the embedded record */\n> +/* frees and zeroes out the embedded record */\n> void reftable_record_release(struct reftable_record *rec);\n\nI didn't follow all the vtable entries, but for these:\n \n 4 matches for \".release =\" in buffer: record.c \n 440: .release = &reftable_ref_record_release_void,\n 582: .release = &reftable_obj_record_release,\n 925: .release = &reftable_log_record_release_void,\n 1052: .release = &reftable_index_record_release,\n\nSome zero'd the data out already, but for\n\"reftable_index_record_release\" isn't promising this a bug, as we don't\nwant to memset() to 0 a strbuf_init()?\n\n> static void test_copy(struct reftable_record *rec)\n> {\n> -\tstruct reftable_record copy =\n> -\t\treftable_new_record(reftable_record_type(rec));\n> +\tstruct reftable_record copy = { 0 };\n> +\tuint8_t typ;\n> +\n> +\ttyp = reftable_record_type(rec);\n\nMore odd s/typ/type/\n\nAlso for the whitespace here we can just add this assignment above where\nwe declare it.\n\n> [...]\n> \t\t};\n> -\n> \t\tint n, m;\n\nGood whitespace change, but another thing to split up in unrelated\nwhitespace fixes...\n\n> [...]\n> \tstruct strbuf key = STRBUF_INIT;\n> -\tstruct reftable_record rec = { NULL };\n> -\tstruct reftable_index_record out = { .last_key = STRBUF_INIT };\n> -\tstruct reftable_record out_rec = { NULL };\n> +\tstruct reftable_record out = {\n> +\t\t.type = BLOCK_TYPE_INDEX,\n> +\t\t.u.idx = { .last_key = STRBUF_INIT },\n> +\t};\n\nAnother case of different struct formatting (but the one I suggested using uniformly).\n\n> \t\tint err = writer_finish_public_section(w);\n> @@ -316,8 +320,6 @@ static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n> \n> \tw->next -= w->pending_padding;\n> \tw->pending_padding = 0;\n> -\n\nstray whitespace change...\n\n\n> \tif (arg->err == 0)\n> \t\tgoto done;\n> -\tobj_rec.offset_len = 0;\n> +\n\n..stray whitespace change...\n\n> +\trec.u.obj.offset_len = 0;\n> \targ->err = block_writer_add(arg->w->block_writer, &rec);\n> \n> \t/* Should be able to write into a fresh block. */\n\n"},{"id":"446629","messageId":"220121.86bl05jm8o.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"82f140cab5c30b59de534d773a5d00343e0868f9.1642691534.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v6 15/15] reftable: add print functions to the record types","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-21T12:33:05Z","receivedAt":"2022-01-21T12:40:14Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jan 20 2022, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n> [...]\n> +void reftable_ref_record_print(const struct reftable_ref_record *ref,\n> +\t\t\t uint32_t hash_id) {\n\n...here...\n\n> +\treftable_ref_record_print_sz(ref, hash_size(hash_id));\n> +}\n> +\n> static void reftable_ref_record_release_void(void *rec)\n> {\n> \treftable_ref_record_release(rec);\n> @@ -443,6 +448,12 @@ static int reftable_ref_record_equal_void(const void *a,\n> \treturn reftable_ref_record_equal(ra, rb, hash_size);\n> }\n> \n> +static void reftable_ref_record_print_void(const void *rec,\n> +\t\t\t\t\t int hash_size)\n\n...and here you avoid a line longer than 79 characters....\n\n> +{\n> +\treftable_ref_record_print_sz((struct reftable_ref_record *) rec, hash_size);\n\n...but not here...\n\n> +\t\tstrbuf_addf(&offset_str, \"%\" PRIu64 \" \", obj->offsets[i]);\n\n\"%\"PRIu64 not \"%\" PRIu64, per the usual coding style....\n\n> -\t\tprintf(\"log{%s(%\" PRIu64 \") delete\", log->refname,\n> +\t\tprintf(\"log{%s(%\" PRIu64 \") delete\\n\", log->refname,\n\n...although I see we have some existing code here in reftable/ using the other style..\n\n> +static void reftable_index_record_print(const void *rec, int hash_size)\n> +{\n> +\tconst struct reftable_index_record *idx = rec;\n> +\t/* TODO: escape null chars? */\n\nThat's quite scary as a TODO comment, can we think about whether we're\ndoing the wrong thing here? I.e. the printf() will stop on a \\0, or\nmaybe I'm misunderstanding this...\n\n> \t/* Are two records equal? This assumes they have the same type. Returns 0 for non-equal. */\n> \tint (*equal)(const void *a, const void *b, int hash_size);\n> +\n> +\t/* Print on stdout, for debugging. */\n\nInaccurate comment, this isn't for ad-hoc debugging, we're using this in\nthe reftable tests....\n\n> +\tvoid (*print)(const void *rec, int hash_size);\n> };\n> \n> /* returns true for recognized block types. Block start with the block type. */\n> @@ -112,6 +115,7 @@ struct reftable_record {\n> \n> /* see struct record_vtable */\n> int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size);\n> +void reftable_record_print(struct reftable_record *rec, int hash_size);\n> void reftable_record_key(struct reftable_record *rec, struct strbuf *dest);\n> uint8_t reftable_record_type(struct reftable_record *rec);\n> void reftable_record_copy_from(struct reftable_record *rec,\n> diff --git a/reftable/record_test.c b/reftable/record_test.c\n> index c6fdd1925a9..f91ea5e8830 100644\n> --- a/reftable/record_test.c\n> +++ b/reftable/record_test.c\n> @@ -25,6 +25,10 @@ static void test_copy(struct reftable_record *rec)\n> \t/* do it twice to catch memory leaks */\n> \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n> \tEXPECT(reftable_record_equal(rec, ©, GIT_SHA1_RAWSZ));\n> +\n> +\tputs(\"testing print coverage:\\n\");\n> +\treftable_record_print(©, GIT_SHA1_RAWSZ);\n\n...i.e. here.\n\n> +\n> \treftable_record_release(©);\n> }\n> \n> @@ -176,7 +180,8 @@ static void test_reftable_log_record_equal(void)\n> static void test_reftable_log_record_roundtrip(void)\n> {\n> \tint i;\n> -\tstruct reftable_log_record in[2] = {\n> +\n\n...more stray whitespace...\n\n> +\tstruct reftable_log_record in[] = {\n> \t\t{\n> \t\t\t.refname = xstrdup(\"refs/heads/master\"),\n> \t\t\t.update_index = 42,\n> @@ -197,10 +202,24 @@ static void test_reftable_log_record_roundtrip(void)\n> \t\t\t.refname = xstrdup(\"refs/heads/master\"),\n> \t\t\t.update_index = 22,\n> \t\t\t.value_type = REFTABLE_LOG_DELETION,\n> +\t\t},\n> +\t\t{\n> +\t\t\t.refname = xstrdup(\"branch\"),\n> +\t\t\t.update_index = 33,\n> +\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n> +\t\t\t.value = {\n> +\t\t\t\t.update = {\n> +\t\t\t\t\t.old_hash = reftable_malloc(GIT_SHA1_RAWSZ),\n> +\t\t\t\t\t.new_hash = reftable_malloc(GIT_SHA1_RAWSZ),\n> +\t\t\t\t\t/* rest of fields left empty. */\n\nBut in test_reftable_log_record_roundtrip() we have another existing\nuser who leaves out fields, but doesn't have such a comment.\n\nI think it's better to just leave it out here, clearly it's intentional,\nbut if it's going to be added to anything surely the other user defining\n5/7 fields is better than this user of 2/7 fields (i.e. the 5/7 is more\nlikely to be an unintentional omission), this one is clearly\nintentional.\n"},{"id":"446670","messageId":"xmqq5yqcd160.fsf@gitster.g","threadId":"57047","inReplyTo":"220121.865yqdl3ff.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v6 02/15] reftable: fix resource leak in block.c error path","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-22T01:11:51Z","receivedAt":"2022-01-22T01:11:56Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> On Thu, Jan 20 2022, Han-Wen Nienhuys via GitGitGadget wrote:\n>\n>> From: Han-Wen Nienhuys \n>>\n>> Add test coverage for corrupt zlib data. Fix memory leaks demonstrated by\n>> unittest.\n>>\n>> This problem was discovered by a Coverity scan.\n>>\n>> Signed-off-by: Han-Wen Nienhuys \n>> ---\n>> reftable/block.c | 26 +++++++++------\n>> reftable/reader.c | 23 ++++++++------\n>> reftable/readwrite_test.c | 66 +++++++++++++++++++++++++++++++++++++++\n>> 3 files changed, 97 insertions(+), 18 deletions(-)\n>>\n>> diff --git a/reftable/block.c b/reftable/block.c\n>> index 855e3f5c947..6c8e8705205 100644\n>> --- a/reftable/block.c\n>> +++ b/reftable/block.c\n>> @@ -188,13 +188,16 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n>> \tuint32_t full_block_size = table_block_size;\n>> \tuint8_t typ = block->data[header_off];\n>> \tuint32_t sz = get_be24(block->data + header_off + 1);\n>> -\n>\n> stray extra whitespace being added in the middle of variable\n> declarations.\n\nHmph. Isn't it removing a blank line?\n"},{"id":"446749","messageId":"CAFQ2z_Pe6PsV4nMRqQRrFTnOQ2+Lb1o-pCn2oBiVAkec2RtkPw@mail.gmail.com","threadId":"57047","inReplyTo":"220121.86a6fpl3hr.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v6 01/15] reftable: fix OOB stack write in print functions","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2022-01-24T14:14:37Z","receivedAt":"2022-01-24T14:14:52Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Fri, Jan 21, 2022 at 12:42 PM Ævar Arnfjörð Bjarmason\n wrote:\n> > - char hex[2 * GIT_SHA256_RAWSZ + 1] = { 0 }; /* BUG */\n> > + char hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */\n>\n> Is whatever \"BUG\" this is still current after this change?\n\nI can't remember what this was about. Dropping.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"446752","messageId":"CAFQ2z_PAqW+RS2Znaf2wwOJfdNfkjP1VV84=xaPu_1EAuX+u5w@mail.gmail.com","threadId":"57047","inReplyTo":"220121.86fsphjml0.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v6 14/15] reftable: make reftable_record a tagged union","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2022-01-24T15:34:41Z","receivedAt":"2022-01-24T15:34:57Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Fri, Jan 21, 2022 at 1:32 PM Ævar Arnfjörð Bjarmason\n wrote:\n> > From: Han-Wen Nienhuys \n> >\n> > This reduces the amount of glue code, because we don't need a void\n> > pointer or vtable within the structure.\n> > [...]\n> > - struct reftable_record rec = { NULL };\n> > - reftable_record_from_ref(&rec, &ref);\n> > + struct reftable_record rec = { .type = BLOCK_TYPE_REF,\n> > + .u.ref = {\n> > + .refname = (char *)name,\n> > + } };\n> > return tab->ops->seek_record(tab->table_arg, it, &rec);\n> > }\n>\n> Both for this & the below don't we prefer to have such assignments on\n> their own line? I.e.:\n\nI generally let clang-format do formatting for me, but it looks like\nit doesn't want to make a decision about where to put line breaks\nhere.\n\nI've made the style consistent.\n> > + struct reftable_record clean = {\n> > + .type = typ,\n>\n> ...only this \"type\" member, which won't confuse a compiler.\n>\n> > + switch (typ) {\n>\n> ...but actually...\n\ntyp vs type is a remnant of the Go origins (type is a reserved word in Go).\n\nI fixed up a couple of places where I add it newly in the commit, but\nit makes more sense to change this throughout in a separate commit.\n\n> > + return clean;\n> > +}\n>\n> ...reading this whole function shouldn't this be a more normal *_init()\n> pattern function where the caller just populates the \".type = \", and we\n> init the rest here? That would also make the ownership more obvious, and\n> if any future API user needs to pass in variable on the heap instead of\n> us returning it on the stack here...\n\nthis function exists so you can initialize as part of a list of\ndeclarations , eg.\n\n struct reftable_record rec = reftable_new_record(block_reader_type(br));\n struct strbuf key = STRBUF_INIT;\n int err = 0;\n struct block_iter next = {\n .last_key = STRBUF_INIT,\n };\n int i = binsearch(br->restart_count, &restart_key_less, &args);\n\nwith init functions, you have reorganize all of these blocks.\n\n\n> > -/* zeroes out the embedded record */\n> > +/* frees and zeroes out the embedded record */\n> > void reftable_record_release(struct reftable_record *rec);\n>\n> I didn't follow all the vtable entries, but for these:\n>\n> 4 matches for \".release =\" in buffer: record.c\n> 440: .release = &reftable_ref_record_release_void,\n> 582: .release = &reftable_obj_record_release,\n> 925: .release = &reftable_log_record_release_void,\n> 1052: .release = &reftable_index_record_release,\n>\n> Some zero'd the data out already, but for\n> \"reftable_index_record_release\" isn't promising this a bug, as we don't\n> want to memset() to 0 a strbuf_init()?\n\nIsn't strbuf_release() -which is used- the correct way to clear out a strbuf?\n\n>> }\n>> -\n>\n>...more stray whitespace...\n\nIf you trim this much context, can you provide a line number?\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"446753","messageId":"CAFQ2z_OsEM3jOs7O5K-bGT3RqidppSzR1NuzeFEJ8_se+Cx6gg@mail.gmail.com","threadId":"57047","inReplyTo":"220121.86bl05jm8o.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v6 15/15] reftable: add print functions to the record types","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2022-01-24T15:50:39Z","receivedAt":"2022-01-24T15:50:55Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Fri, Jan 21, 2022 at 1:40 PM Ævar Arnfjörð Bjarmason\n wrote:\n> > +static void reftable_index_record_print(const void *rec, int hash_size)\n> > +{\n> > + const struct reftable_index_record *idx = rec;\n> > + /* TODO: escape null chars? */\n>\n> That's quite scary as a TODO comment, can we think about whether we're\n> doing the wrong thing here? I.e. the printf() will stop on a \\0, or\n> maybe I'm misunderstanding this...\n\nIt's a print function, so we'd print a truncated last_key. This can\nhappen if we were printing an index record that points to a log\nrecord. I'd address this if and when I have to debug into indexing\ngoing wrong (which hasn't happened so far.)\n\n\n> > /* Are two records equal? This assumes they have the same type. Returns 0 for non-equal. */\n> > int (*equal)(const void *a, const void *b, int hash_size);\n> > +\n> > + /* Print on stdout, for debugging. */\n>\n> Inaccurate comment, this isn't for ad-hoc debugging, we're using this in\n> the reftable tests....\n\nThe tests say\n\n> > + puts(\"testing print coverage:\\n\");\n> > + reftable_record_print(©, GIT_SHA1_RAWSZ);\n\nie. I'm calling the print functions to make sure they don't crash or\nleak memory. They're not used as part of functionality of either the\nreftable code or its tests.\n\n> > + .old_hash = reftable_malloc(GIT_SHA1_RAWSZ),\n> > + .new_hash = reftable_malloc(GIT_SHA1_RAWSZ),\n> > + /* rest of fields left empty. */\n>\n> But in test_reftable_log_record_roundtrip() we have another existing\n> user who leaves out fields, but doesn't have such a comment.\n\nrephrased comment.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\nRegistergericht und -nummer: Hamburg, HRB 86891\nSitz der Gesellschaft: Hamburg\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"446782","messageId":"b0bffb6e29d2d060c7698cd3e84918f14ad8a3e3.1643051624.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v7.git.git.1643051624.gitgitgadget@gmail.com","subject":"[PATCH v7 04/16] reftable: check reftable_stack_auto_compact() return value","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-24T19:13:32Z","receivedAt":"2022-01-24T19:32:40Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nIf compaction consistently errors out, we would accumulate reftables,\nand the follow-on EXPECT would start failing, but catching errors\nearly helps debugging.\n\nSpotted by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack_test.c | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/reftable/stack_test.c b/reftable/stack_test.c\nindex f4c743db80c..e84f50d27ff 100644\n--- a/reftable/stack_test.c\n+++ b/reftable/stack_test.c\n@@ -839,6 +839,7 @@ static void test_reftable_stack_auto_compaction(void)\n \t\tEXPECT_ERR(err);\n \n \t\terr = reftable_stack_auto_compact(st);\n+\t\tEXPECT_ERR(err);\n \t\tEXPECT(i < 3 || st->merged->stack_len < 2 * fastlog2(i));\n \t}\n \n-- \ngitgitgadget\n\n"},{"id":"446783","messageId":"a83a9cf59b7b3e7200611e067b59866d0b678b46.1643051624.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v7.git.git.1643051624.gitgitgadget@gmail.com","subject":"[PATCH v7 12/16] reftable: implement record equality generically","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-24T19:13:40Z","receivedAt":"2022-01-24T19:33:05Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis simplifies unittests a little, and provides further coverage for\nreftable_record_copy().\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 54 +++++++++++++++++++++++++++++++++++++++++-\n reftable/record.h | 5 +++-\n reftable/record_test.c | 23 +++---------------\n 3 files changed, 60 insertions(+), 22 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 1ae344c745b..b543585ad39 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -430,6 +430,14 @@ static int reftable_ref_record_is_deletion_void(const void *p)\n \t\t(const struct reftable_ref_record *)p);\n }\n \n+static int reftable_ref_record_equal_void(const void *a,\n+\t\t\t\t\t const void *b, int hash_size)\n+{\n+\tstruct reftable_ref_record *ra = (struct reftable_ref_record *) a;\n+\tstruct reftable_ref_record *rb = (struct reftable_ref_record *) b;\n+\treturn reftable_ref_record_equal(ra, rb, hash_size);\n+}\n+\n static struct reftable_record_vtable reftable_ref_record_vtable = {\n \t.key = &reftable_ref_record_key,\n \t.type = BLOCK_TYPE_REF,\n@@ -439,6 +447,7 @@ static struct reftable_record_vtable reftable_ref_record_vtable = {\n \t.decode = &reftable_ref_record_decode,\n \t.release = &reftable_ref_record_release_void,\n \t.is_deletion = &reftable_ref_record_is_deletion_void,\n+\t.equal = &reftable_ref_record_equal_void,\n };\n \n static void reftable_obj_record_key(const void *r, struct strbuf *dest)\n@@ -572,6 +581,23 @@ static int not_a_deletion(const void *p)\n \treturn 0;\n }\n \n+static int reftable_obj_record_equal_void(const void *a, const void *b, int hash_size)\n+{\n+\tstruct reftable_obj_record *ra = (struct reftable_obj_record *) a;\n+\tstruct reftable_obj_record *rb = (struct reftable_obj_record *) b;\n+\n+\tif (ra->hash_prefix_len != rb->hash_prefix_len\n+\t || ra->offset_len != rb->offset_len)\n+\t\treturn 0;\n+\n+\tif (memcmp(ra->hash_prefix, rb->hash_prefix, ra->hash_prefix_len))\n+\t\treturn 0;\n+\tif (memcmp(ra->offsets, rb->offsets, ra->offset_len * sizeof(uint64_t)))\n+\t\treturn 0;\n+\n+\treturn 1;\n+}\n+\n static struct reftable_record_vtable reftable_obj_record_vtable = {\n \t.key = &reftable_obj_record_key,\n \t.type = BLOCK_TYPE_OBJ,\n@@ -580,7 +606,8 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {\n \t.encode = &reftable_obj_record_encode,\n \t.decode = &reftable_obj_record_decode,\n \t.release = &reftable_obj_record_release,\n-\t.is_deletion = not_a_deletion,\n+\t.is_deletion = ¬_a_deletion,\n+\t.equal = &reftable_obj_record_equal_void,\n };\n \n void reftable_log_record_print(struct reftable_log_record *log,\n@@ -881,6 +908,14 @@ static int zero_hash_eq(uint8_t *a, uint8_t *b, int sz)\n \treturn !memcmp(a, b, sz);\n }\n \n+static int reftable_log_record_equal_void(const void *a,\n+\t\t\t\t\t const void *b, int hash_size)\n+{\n+\treturn reftable_log_record_equal((struct reftable_log_record *) a,\n+\t\t\t\t\t (struct reftable_log_record *) b,\n+\t\t\t\t\t hash_size);\n+}\n+\n int reftable_log_record_equal(const struct reftable_log_record *a,\n \t\t\t const struct reftable_log_record *b, int hash_size)\n {\n@@ -924,6 +959,7 @@ static struct reftable_record_vtable reftable_log_record_vtable = {\n \t.decode = &reftable_log_record_decode,\n \t.release = &reftable_log_record_release_void,\n \t.is_deletion = &reftable_log_record_is_deletion_void,\n+\t.equal = &reftable_log_record_equal_void\n };\n \n struct reftable_record reftable_new_record(uint8_t typ)\n@@ -1042,6 +1078,14 @@ static int reftable_index_record_decode(void *rec, struct strbuf key,\n \treturn start.len - in.len;\n }\n \n+static int reftable_index_record_equal(const void *a, const void *b, int hash_size)\n+{\n+\tstruct reftable_index_record *ia = (struct reftable_index_record *) a;\n+\tstruct reftable_index_record *ib = (struct reftable_index_record *) b;\n+\n+\treturn ia->offset == ib->offset && !strbuf_cmp(&ia->last_key, &ib->last_key);\n+}\n+\n static struct reftable_record_vtable reftable_index_record_vtable = {\n \t.key = &reftable_index_record_key,\n \t.type = BLOCK_TYPE_INDEX,\n@@ -1051,6 +1095,7 @@ static struct reftable_record_vtable reftable_index_record_vtable = {\n \t.decode = &reftable_index_record_decode,\n \t.release = &reftable_index_record_release,\n \t.is_deletion = ¬_a_deletion,\n+\t.equal = &reftable_index_record_equal,\n };\n \n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)\n@@ -1098,6 +1143,13 @@ int reftable_record_is_deletion(struct reftable_record *rec)\n \treturn rec->ops->is_deletion(rec->data);\n }\n \n+int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size)\n+{\n+\tif (a->ops != b->ops)\n+\t\treturn 0;\n+\treturn a->ops->equal(a->data, b->data, hash_size);\n+}\n+\n void reftable_record_from_ref(struct reftable_record *rec,\n \t\t\t struct reftable_ref_record *ref_rec)\n {\ndiff --git a/reftable/record.h b/reftable/record.h\nindex 498e8c50bf4..da75d7d1f11 100644\n--- a/reftable/record.h\n+++ b/reftable/record.h\n@@ -58,6 +58,9 @@ struct reftable_record_vtable {\n \n \t/* is this a tombstone? */\n \tint (*is_deletion)(const void *rec);\n+\n+\t/* Are two records equal? This assumes they have the same type. Returns 0 for non-equal. */\n+\tint (*equal)(const void *a, const void *b, int hash_size);\n };\n \n /* record is a generic wrapper for different types of records. */\n@@ -98,7 +101,7 @@ struct reftable_obj_record {\n };\n \n /* see struct record_vtable */\n-\n+int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size);\n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest);\n uint8_t reftable_record_type(struct reftable_record *rec);\n void reftable_record_copy_from(struct reftable_record *rec,\ndiff --git a/reftable/record_test.c b/reftable/record_test.c\nindex f4ad7cace41..92680848156 100644\n--- a/reftable/record_test.c\n+++ b/reftable/record_test.c\n@@ -21,18 +21,7 @@ static void test_copy(struct reftable_record *rec)\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \t/* do it twice to catch memory leaks */\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n-\tswitch (reftable_record_type(©)) {\n-\tcase BLOCK_TYPE_REF:\n-\t\tEXPECT(reftable_ref_record_equal(reftable_record_as_ref(©),\n-\t\t\t\t\t\t reftable_record_as_ref(rec),\n-\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n-\t\tbreak;\n-\tcase BLOCK_TYPE_LOG:\n-\t\tEXPECT(reftable_log_record_equal(reftable_record_as_log(©),\n-\t\t\t\t\t\t reftable_record_as_log(rec),\n-\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n-\t\tbreak;\n-\t}\n+\tEXPECT(reftable_record_equal(rec, ©, GIT_SHA1_RAWSZ));\n \treftable_record_destroy(©);\n }\n \n@@ -346,13 +335,7 @@ static void test_reftable_obj_record_roundtrip(void)\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(in.hash_prefix_len == out.hash_prefix_len);\n-\t\tEXPECT(in.offset_len == out.offset_len);\n-\n-\t\tEXPECT(!memcmp(in.hash_prefix, out.hash_prefix,\n-\t\t\t in.hash_prefix_len));\n-\t\tEXPECT(0 == memcmp(in.offsets, out.offsets,\n-\t\t\t\t sizeof(uint64_t) * in.offset_len));\n+\t\tEXPECT(reftable_record_equal(&rec, &rec_out, GIT_SHA1_RAWSZ));\n \t\tstrbuf_release(&key);\n \t\treftable_record_release(&rec_out);\n \t}\n@@ -390,7 +373,7 @@ static void test_reftable_index_record_roundtrip(void)\n \tm = reftable_record_decode(&out_rec, key, extra, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(m == n);\n \n-\tEXPECT(in.offset == out.offset);\n+\tEXPECT(reftable_record_equal(&rec, &out_rec, GIT_SHA1_RAWSZ));\n \n \treftable_record_release(&out_rec);\n \tstrbuf_release(&key);\n-- \ngitgitgadget\n\n"},{"id":"446784","messageId":"7e23d14d1af8624067c0afd8f21e793ee551a78f.1643051624.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v7.git.git.1643051624.gitgitgadget@gmail.com","subject":"[PATCH v7 13/16] reftable: remove outdated file reftable.c","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-24T19:13:41Z","receivedAt":"2022-01-24T19:33:07Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis was renamed to generic.c, but the origin was never removed\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/reftable.c | 115 --------------------------------------------\n 1 file changed, 115 deletions(-)\n delete mode 100644 reftable/reftable.c\n\ndiff --git a/reftable/reftable.c b/reftable/reftable.c\ndeleted file mode 100644\nindex 0e4607a7cd6..00000000000\n--- a/reftable/reftable.c\n+++ /dev/null\n@@ -1,115 +0,0 @@\n-/*\n-Copyright 2020 Google LLC\n-\n-Use of this source code is governed by a BSD-style\n-license that can be found in the LICENSE file or at\n-https://developers.google.com/open-source/licenses/bsd\n-*/\n-\n-#include \"basics.h\"\n-#include \"record.h\"\n-#include \"generic.h\"\n-#include \"reftable-iterator.h\"\n-#include \"reftable-generic.h\"\n-\n-int reftable_table_seek_ref(struct reftable_table *tab,\n-\t\t\t struct reftable_iterator *it, const char *name)\n-{\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n-\t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n-\treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n-}\n-\n-int reftable_table_read_ref(struct reftable_table *tab, const char *name,\n-\t\t\t struct reftable_ref_record *ref)\n-{\n-\tstruct reftable_iterator it = { NULL };\n-\tint err = reftable_table_seek_ref(tab, &it, name);\n-\tif (err)\n-\t\tgoto done;\n-\n-\terr = reftable_iterator_next_ref(&it, ref);\n-\tif (err)\n-\t\tgoto done;\n-\n-\tif (strcmp(ref->refname, name) ||\n-\t reftable_ref_record_is_deletion(ref)) {\n-\t\treftable_ref_record_release(ref);\n-\t\terr = 1;\n-\t\tgoto done;\n-\t}\n-\n-done:\n-\treftable_iterator_destroy(&it);\n-\treturn err;\n-}\n-\n-uint64_t reftable_table_max_update_index(struct reftable_table *tab)\n-{\n-\treturn tab->ops->max_update_index(tab->table_arg);\n-}\n-\n-uint64_t reftable_table_min_update_index(struct reftable_table *tab)\n-{\n-\treturn tab->ops->min_update_index(tab->table_arg);\n-}\n-\n-uint32_t reftable_table_hash_id(struct reftable_table *tab)\n-{\n-\treturn tab->ops->hash_id(tab->table_arg);\n-}\n-\n-void reftable_iterator_destroy(struct reftable_iterator *it)\n-{\n-\tif (!it->ops) {\n-\t\treturn;\n-\t}\n-\tit->ops->close(it->iter_arg);\n-\tit->ops = NULL;\n-\tFREE_AND_NULL(it->iter_arg);\n-}\n-\n-int reftable_iterator_next_ref(struct reftable_iterator *it,\n-\t\t\t struct reftable_ref_record *ref)\n-{\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, ref);\n-\treturn iterator_next(it, &rec);\n-}\n-\n-int reftable_iterator_next_log(struct reftable_iterator *it,\n-\t\t\t struct reftable_log_record *log)\n-{\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, log);\n-\treturn iterator_next(it, &rec);\n-}\n-\n-int iterator_next(struct reftable_iterator *it, struct reftable_record *rec)\n-{\n-\treturn it->ops->next(it->iter_arg, rec);\n-}\n-\n-static int empty_iterator_next(void *arg, struct reftable_record *rec)\n-{\n-\treturn 1;\n-}\n-\n-static void empty_iterator_close(void *arg)\n-{\n-}\n-\n-static struct reftable_iterator_vtable empty_vtable = {\n-\t.next = &empty_iterator_next,\n-\t.close = &empty_iterator_close,\n-};\n-\n-void iterator_set_empty(struct reftable_iterator *it)\n-{\n-\tassert(!it->ops);\n-\tit->iter_arg = NULL;\n-\tit->ops = &empty_vtable;\n-}\n-- \ngitgitgadget\n\n"},{"id":"446785","messageId":"a118cd40975e5844050af1a643140a915f9639f6.1643051624.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v7.git.git.1643051624.gitgitgadget@gmail.com","subject":"[PATCH v7 09/16] reftable: drop stray printf in readwrite_test","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-24T19:13:37Z","receivedAt":"2022-01-24T19:33:10Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/readwrite_test.c | 1 -\n 1 file changed, 1 deletion(-)\n\ndiff --git a/reftable/readwrite_test.c b/reftable/readwrite_test.c\nindex edb9cfc1930..605ba0f9fd4 100644\n--- a/reftable/readwrite_test.c\n+++ b/reftable/readwrite_test.c\n@@ -696,7 +696,6 @@ static void test_write_key_order(void)\n \terr = reftable_writer_add_ref(w, &refs[0]);\n \tEXPECT_ERR(err);\n \terr = reftable_writer_add_ref(w, &refs[1]);\n-\tprintf(\"%d\\n\", err);\n \tEXPECT(err == REFTABLE_API_ERROR);\n \treftable_writer_close(w);\n \treftable_writer_free(w);\n-- \ngitgitgadget\n\n"},{"id":"446786","messageId":"556fc11080a67ceda81f6c39fcdd4dc1098ac1b5.1643051624.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v7.git.git.1643051624.gitgitgadget@gmail.com","subject":"[PATCH v7 16/16] reftable: rename typ to type","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-24T19:13:44Z","receivedAt":"2022-01-24T19:33:12Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThe 'typ' variable name betrays the Go origins of this code (in Go,\n'type' is a reserved word). Since we now have parted ways with the Go\ncode, simply use 'type'.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/block.c | 10 ++++----\n reftable/merged.c | 6 ++---\n reftable/merged.h | 2 +-\n reftable/reader.c | 58 +++++++++++++++++++++++------------------------\n reftable/record.c | 4 ++--\n reftable/writer.c | 32 +++++++++++++-------------\n 6 files changed, 56 insertions(+), 56 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 80ad10bb794..6c1f70ba563 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -68,14 +68,14 @@ static int block_writer_register_restart(struct block_writer *w, int n,\n \treturn 0;\n }\n \n-void block_writer_init(struct block_writer *bw, uint8_t typ, uint8_t *buf,\n+void block_writer_init(struct block_writer *bw, uint8_t type, uint8_t *buf,\n \t\t uint32_t block_size, uint32_t header_off, int hash_size)\n {\n \tbw->buf = buf;\n \tbw->hash_size = hash_size;\n \tbw->block_size = block_size;\n \tbw->header_off = header_off;\n-\tbw->buf[header_off] = typ;\n+\tbw->buf[header_off] = type;\n \tbw->next = header_off + 4;\n \tbw->restart_interval = 16;\n \tbw->entries = 0;\n@@ -186,7 +186,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\t int hash_size)\n {\n \tuint32_t full_block_size = table_block_size;\n-\tuint8_t typ = block->data[header_off];\n+\tuint8_t type = block->data[header_off];\n \tuint32_t sz = get_be24(block->data + header_off + 1);\n \n \tint err = 0;\n@@ -195,12 +195,12 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tuint8_t *restart_bytes = NULL;\n \tuint8_t *uncompressed = NULL;\n \n-\tif (!reftable_is_block_type(typ)) {\n+\tif (!reftable_is_block_type(type)) {\n \t\terr = REFTABLE_FORMAT_ERROR;\n \t\tgoto done;\n \t}\n \n-\tif (typ == BLOCK_TYPE_LOG) {\n+\tif (type == BLOCK_TYPE_LOG) {\n \t\tint block_header_skip = 4 + header_off;\n \t\tuLongf dst_len = sz - block_header_skip; /* total size of dest\n \t\t\t\t\t\t\t buffer. */\ndiff --git a/reftable/merged.c b/reftable/merged.c\nindex 82511762d2a..b5710d5f7a9 100644\n--- a/reftable/merged.c\n+++ b/reftable/merged.c\n@@ -22,7 +22,7 @@ static int merged_iter_init(struct merged_iter *mi)\n {\n \tint i = 0;\n \tfor (i = 0; i < mi->stack_len; i++) {\n-\t\tstruct reftable_record rec = reftable_new_record(mi->typ);\n+\t\tstruct reftable_record rec = reftable_new_record(mi->type);\n \t\tint err = iterator_next(&mi->stack[i], &rec);\n \t\tif (err < 0) {\n \t\t\treturn err;\n@@ -58,7 +58,7 @@ static int merged_iter_advance_nonnull_subiter(struct merged_iter *mi,\n \t\t\t\t\t size_t idx)\n {\n \tstruct pq_entry e = {\n-\t\t.rec = reftable_new_record(mi->typ),\n+\t\t.rec = reftable_new_record(mi->type),\n \t\t.index = idx,\n \t};\n \tint err = iterator_next(&mi->stack[idx], &e.rec);\n@@ -245,7 +245,7 @@ static int merged_table_seek_record(struct reftable_merged_table *mt,\n \t\tsizeof(struct reftable_iterator) * mt->stack_len);\n \tstruct merged_iter merged = {\n \t\t.stack = iters,\n-\t\t.typ = reftable_record_type(rec),\n+\t\t.type = reftable_record_type(rec),\n \t\t.hash_id = mt->hash_id,\n \t\t.suppress_deletions = mt->suppress_deletions,\n \t};\ndiff --git a/reftable/merged.h b/reftable/merged.h\nindex 7d9f95d27ed..63f288ff8d6 100644\n--- a/reftable/merged.h\n+++ b/reftable/merged.h\n@@ -28,7 +28,7 @@ struct merged_iter {\n \tstruct reftable_iterator *stack;\n \tuint32_t hash_id;\n \tsize_t stack_len;\n-\tuint8_t typ;\n+\tuint8_t type;\n \tint suppress_deletions;\n \tstruct merged_iter_pqueue pq;\n };\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 565cda8d5a2..1ecd23bba36 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -43,9 +43,9 @@ void block_source_close(struct reftable_block_source *source)\n }\n \n static struct reftable_reader_offsets *\n-reader_offsets_for(struct reftable_reader *r, uint8_t typ)\n+reader_offsets_for(struct reftable_reader *r, uint8_t type)\n {\n-\tswitch (typ) {\n+\tswitch (type) {\n \tcase BLOCK_TYPE_REF:\n \t\treturn &r->ref_offsets;\n \tcase BLOCK_TYPE_LOG:\n@@ -84,7 +84,7 @@ static int parse_footer(struct reftable_reader *r, uint8_t *footer,\n \t\t\tuint8_t *header)\n {\n \tuint8_t *f = footer;\n-\tuint8_t first_block_typ;\n+\tuint8_t first_block_type;\n \tint err = 0;\n \tuint32_t computed_crc;\n \tuint32_t file_crc;\n@@ -149,10 +149,10 @@ static int parse_footer(struct reftable_reader *r, uint8_t *footer,\n \t\tgoto done;\n \t}\n \n-\tfirst_block_typ = header[header_size(r->version)];\n-\tr->ref_offsets.is_present = (first_block_typ == BLOCK_TYPE_REF);\n+\tfirst_block_type = header[header_size(r->version)];\n+\tr->ref_offsets.is_present = (first_block_type == BLOCK_TYPE_REF);\n \tr->ref_offsets.offset = 0;\n-\tr->log_offsets.is_present = (first_block_typ == BLOCK_TYPE_LOG ||\n+\tr->log_offsets.is_present = (first_block_type == BLOCK_TYPE_LOG ||\n \t\t\t\t r->log_offsets.offset > 0);\n \tr->obj_offsets.is_present = r->obj_offsets.offset > 0;\n \terr = 0;\n@@ -214,7 +214,7 @@ done:\n \n struct table_iter {\n \tstruct reftable_reader *r;\n-\tuint8_t typ;\n+\tuint8_t type;\n \tuint64_t block_off;\n \tstruct block_iter bi;\n \tint is_finished;\n@@ -228,7 +228,7 @@ static void table_iter_copy_from(struct table_iter *dest,\n \t\t\t\t struct table_iter *src)\n {\n \tdest->r = src->r;\n-\tdest->typ = src->typ;\n+\tdest->type = src->type;\n \tdest->block_off = src->block_off;\n \tdest->is_finished = src->is_finished;\n \tblock_iter_copy_from(&dest->bi, &src->bi);\n@@ -257,7 +257,7 @@ static void table_iter_block_done(struct table_iter *ti)\n \tti->bi.next_off = 0;\n }\n \n-static int32_t extract_block_size(uint8_t *data, uint8_t *typ, uint64_t off,\n+static int32_t extract_block_size(uint8_t *data, uint8_t *type, uint64_t off,\n \t\t\t\t int version)\n {\n \tint32_t result = 0;\n@@ -266,20 +266,20 @@ static int32_t extract_block_size(uint8_t *data, uint8_t *typ, uint64_t off,\n \t\tdata += header_size(version);\n \t}\n \n-\t*typ = data[0];\n-\tif (reftable_is_block_type(*typ)) {\n+\t*type = data[0];\n+\tif (reftable_is_block_type(*type)) {\n \t\tresult = get_be24(data + 1);\n \t}\n \treturn result;\n }\n \n int reader_init_block_reader(struct reftable_reader *r, struct block_reader *br,\n-\t\t\t uint64_t next_off, uint8_t want_typ)\n+\t\t\t uint64_t next_off, uint8_t want_type)\n {\n \tint32_t guess_block_size = r->block_size ? r->block_size :\n \t\t\t\t\t\t\t DEFAULT_BLOCK_SIZE;\n \tstruct reftable_block block = { NULL };\n-\tuint8_t block_typ = 0;\n+\tuint8_t block_type = 0;\n \tint err = 0;\n \tuint32_t header_off = next_off ? 0 : header_size(r->version);\n \tint32_t block_size = 0;\n@@ -291,13 +291,13 @@ int reader_init_block_reader(struct reftable_reader *r, struct block_reader *br,\n \tif (err < 0)\n \t\tgoto done;\n \n-\tblock_size = extract_block_size(block.data, &block_typ, next_off,\n+\tblock_size = extract_block_size(block.data, &block_type, next_off,\n \t\t\t\t\tr->version);\n \tif (block_size < 0) {\n \t\terr = block_size;\n \t\tgoto done;\n \t}\n-\tif (want_typ != BLOCK_TYPE_ANY && block_typ != want_typ) {\n+\tif (want_type != BLOCK_TYPE_ANY && block_type != want_type) {\n \t\terr = 1;\n \t\tgoto done;\n \t}\n@@ -326,10 +326,10 @@ static int table_iter_next_block(struct table_iter *dest,\n \tint err = 0;\n \n \tdest->r = src->r;\n-\tdest->typ = src->typ;\n+\tdest->type = src->type;\n \tdest->block_off = next_block_off;\n \n-\terr = reader_init_block_reader(src->r, &br, next_block_off, src->typ);\n+\terr = reader_init_block_reader(src->r, &br, next_block_off, src->type);\n \tif (err > 0) {\n \t\tdest->is_finished = 1;\n \t\treturn 1;\n@@ -349,7 +349,7 @@ static int table_iter_next_block(struct table_iter *dest,\n \n static int table_iter_next(struct table_iter *ti, struct reftable_record *rec)\n {\n-\tif (reftable_record_type(rec) != ti->typ)\n+\tif (reftable_record_type(rec) != ti->type)\n \t\treturn REFTABLE_API_ERROR;\n \n \twhile (1) {\n@@ -404,38 +404,38 @@ static void iterator_from_table_iter(struct reftable_iterator *it,\n \n static int reader_table_iter_at(struct reftable_reader *r,\n \t\t\t\tstruct table_iter *ti, uint64_t off,\n-\t\t\t\tuint8_t typ)\n+\t\t\t\tuint8_t type)\n {\n \tstruct block_reader br = { 0 };\n \tstruct block_reader *brp = NULL;\n \n-\tint err = reader_init_block_reader(r, &br, off, typ);\n+\tint err = reader_init_block_reader(r, &br, off, type);\n \tif (err != 0)\n \t\treturn err;\n \n \tbrp = reftable_malloc(sizeof(struct block_reader));\n \t*brp = br;\n \tti->r = r;\n-\tti->typ = block_reader_type(brp);\n+\tti->type = block_reader_type(brp);\n \tti->block_off = off;\n \tblock_reader_start(brp, &ti->bi);\n \treturn 0;\n }\n \n static int reader_start(struct reftable_reader *r, struct table_iter *ti,\n-\t\t\tuint8_t typ, int index)\n+\t\t\tuint8_t type, int index)\n {\n-\tstruct reftable_reader_offsets *offs = reader_offsets_for(r, typ);\n+\tstruct reftable_reader_offsets *offs = reader_offsets_for(r, type);\n \tuint64_t off = offs->offset;\n \tif (index) {\n \t\toff = offs->index_offset;\n \t\tif (off == 0) {\n \t\t\treturn 1;\n \t\t}\n-\t\ttyp = BLOCK_TYPE_INDEX;\n+\t\ttype = BLOCK_TYPE_INDEX;\n \t}\n \n-\treturn reader_table_iter_at(r, ti, off, typ);\n+\treturn reader_table_iter_at(r, ti, off, type);\n }\n \n static int reader_seek_linear(struct reftable_reader *r, struct table_iter *ti,\n@@ -522,12 +522,12 @@ static int reader_seek_indexed(struct reftable_reader *r,\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n-\t\tif (next.typ == reftable_record_type(rec)) {\n+\t\tif (next.type == reftable_record_type(rec)) {\n \t\t\terr = 0;\n \t\t\tbreak;\n \t\t}\n \n-\t\tif (next.typ != BLOCK_TYPE_INDEX) {\n+\t\tif (next.type != BLOCK_TYPE_INDEX) {\n \t\t\terr = REFTABLE_FORMAT_ERROR;\n \t\t\tbreak;\n \t\t}\n@@ -582,9 +582,9 @@ static int reader_seek_internal(struct reftable_reader *r,\n static int reader_seek(struct reftable_reader *r, struct reftable_iterator *it,\n \t\t struct reftable_record *rec)\n {\n-\tuint8_t typ = reftable_record_type(rec);\n+\tuint8_t type = reftable_record_type(rec);\n \n-\tstruct reftable_reader_offsets *offs = reader_offsets_for(r, typ);\n+\tstruct reftable_reader_offsets *offs = reader_offsets_for(r, type);\n \tif (!offs->is_present) {\n \t\titerator_set_empty(it);\n \t\treturn 0;\ndiff --git a/reftable/record.c b/reftable/record.c\nindex c57bac0e96a..8330b7a2589 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -78,9 +78,9 @@ int put_var_int(struct string_view *dest, uint64_t val)\n \treturn n;\n }\n \n-int reftable_is_block_type(uint8_t typ)\n+int reftable_is_block_type(uint8_t type)\n {\n-\tswitch (typ) {\n+\tswitch (type) {\n \tcase BLOCK_TYPE_REF:\n \tcase BLOCK_TYPE_LOG:\n \tcase BLOCK_TYPE_OBJ:\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex 96a5155ea77..5b768438625 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -26,9 +26,9 @@ static void writer_clear_index(struct reftable_writer *w);\n static int writer_finish_public_section(struct reftable_writer *w);\n \n static struct reftable_block_stats *\n-writer_reftable_block_stats(struct reftable_writer *w, uint8_t typ)\n+writer_reftable_block_stats(struct reftable_writer *w, uint8_t type)\n {\n-\tswitch (typ) {\n+\tswitch (type) {\n \tcase 'r':\n \t\treturn &w->stats.ref_stats;\n \tcase 'o':\n@@ -102,7 +102,7 @@ static int writer_write_header(struct reftable_writer *w, uint8_t *dest)\n \treturn header_size(writer_version(w));\n }\n \n-static void writer_reinit_block_writer(struct reftable_writer *w, uint8_t typ)\n+static void writer_reinit_block_writer(struct reftable_writer *w, uint8_t type)\n {\n \tint block_start = 0;\n \tif (w->next == 0) {\n@@ -110,7 +110,7 @@ static void writer_reinit_block_writer(struct reftable_writer *w, uint8_t typ)\n \t}\n \n \tstrbuf_release(&w->last_key);\n-\tblock_writer_init(&w->block_writer_data, typ, w->block,\n+\tblock_writer_init(&w->block_writer_data, type, w->block,\n \t\t\t w->opts.block_size, block_start,\n \t\t\t hash_size(w->opts.hash_id));\n \tw->block_writer = &w->block_writer_data;\n@@ -375,7 +375,7 @@ int reftable_writer_add_logs(struct reftable_writer *w,\n \n static int writer_finish_section(struct reftable_writer *w)\n {\n-\tuint8_t typ = block_writer_type(w->block_writer);\n+\tuint8_t type = block_writer_type(w->block_writer);\n \tuint64_t index_start = 0;\n \tint max_level = 0;\n \tint threshold = w->opts.unpadded ? 1 : 3;\n@@ -434,7 +434,7 @@ static int writer_finish_section(struct reftable_writer *w)\n \tif (err < 0)\n \t\treturn err;\n \n-\tbstats = writer_reftable_block_stats(w, typ);\n+\tbstats = writer_reftable_block_stats(w, type);\n \tbstats->index_blocks = w->stats.idx_stats.blocks - before_blocks;\n \tbstats->index_offset = index_start;\n \tbstats->max_index_level = max_level;\n@@ -536,17 +536,17 @@ static int writer_dump_object_index(struct reftable_writer *w)\n \n static int writer_finish_public_section(struct reftable_writer *w)\n {\n-\tuint8_t typ = 0;\n+\tuint8_t type = 0;\n \tint err = 0;\n \n \tif (w->block_writer == NULL)\n \t\treturn 0;\n \n-\ttyp = block_writer_type(w->block_writer);\n+\ttype = block_writer_type(w->block_writer);\n \terr = writer_finish_section(w);\n \tif (err < 0)\n \t\treturn err;\n-\tif (typ == BLOCK_TYPE_REF && !w->opts.skip_index_objects &&\n+\tif (type == BLOCK_TYPE_REF && !w->opts.skip_index_objects &&\n \t w->stats.ref_stats.index_blocks > 0) {\n \t\terr = writer_dump_object_index(w);\n \t\tif (err < 0)\n@@ -630,10 +630,10 @@ static const int debug = 0;\n \n static int writer_flush_nonempty_block(struct reftable_writer *w)\n {\n-\tuint8_t typ = block_writer_type(w->block_writer);\n+\tuint8_t type = block_writer_type(w->block_writer);\n \tstruct reftable_block_stats *bstats =\n-\t\twriter_reftable_block_stats(w, typ);\n-\tuint64_t block_typ_off = (bstats->blocks == 0) ? w->next : 0;\n+\t\twriter_reftable_block_stats(w, type);\n+\tuint64_t block_type_off = (bstats->blocks == 0) ? w->next : 0;\n \tint raw_bytes = block_writer_finish(w->block_writer);\n \tint padding = 0;\n \tint err = 0;\n@@ -641,12 +641,12 @@ static int writer_flush_nonempty_block(struct reftable_writer *w)\n \tif (raw_bytes < 0)\n \t\treturn raw_bytes;\n \n-\tif (!w->opts.unpadded && typ != BLOCK_TYPE_LOG) {\n+\tif (!w->opts.unpadded && type != BLOCK_TYPE_LOG) {\n \t\tpadding = w->opts.block_size - raw_bytes;\n \t}\n \n-\tif (block_typ_off > 0) {\n-\t\tbstats->offset = block_typ_off;\n+\tif (block_type_off > 0) {\n+\t\tbstats->offset = block_type_off;\n \t}\n \n \tbstats->entries += w->block_writer->entries;\n@@ -655,7 +655,7 @@ static int writer_flush_nonempty_block(struct reftable_writer *w)\n \tw->stats.blocks++;\n \n \tif (debug) {\n-\t\tfprintf(stderr, \"block %c off %\" PRIu64 \" sz %d (%d)\\n\", typ,\n+\t\tfprintf(stderr, \"block %c off %\" PRIu64 \" sz %d (%d)\\n\", type,\n \t\t\tw->next, raw_bytes,\n \t\t\tget_be24(w->block + w->block_writer->header_off + 1));\n \t}\n-- \ngitgitgadget\n"},{"id":"446787","messageId":"1c5441d2f7e81307550194f44d235174fee49441.1643051624.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v7.git.git.1643051624.gitgitgadget@gmail.com","subject":"[PATCH v7 02/16] reftable: fix resource leak in block.c error path","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-24T19:13:30Z","receivedAt":"2022-01-24T19:38:34Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nAdd test coverage for corrupt zlib data. Fix memory leaks demonstrated by\nunittest.\n\nThis problem was discovered by a Coverity scan.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/block.c | 25 ++++++++++-----\n reftable/reader.c | 23 ++++++++------\n reftable/readwrite_test.c | 66 +++++++++++++++++++++++++++++++++++++++\n 3 files changed, 97 insertions(+), 17 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 855e3f5c947..5832f85501d 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -189,12 +189,16 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tuint8_t typ = block->data[header_off];\n \tuint32_t sz = get_be24(block->data + header_off + 1);\n \n+\tint err = 0;\n \tuint16_t restart_count = 0;\n \tuint32_t restart_start = 0;\n \tuint8_t *restart_bytes = NULL;\n+\tuint8_t *uncompressed = NULL;\n \n-\tif (!reftable_is_block_type(typ))\n-\t\treturn REFTABLE_FORMAT_ERROR;\n+\tif (!reftable_is_block_type(typ)) {\n+\t\terr = REFTABLE_FORMAT_ERROR;\n+\t\tgoto done;\n+\t}\n \n \tif (typ == BLOCK_TYPE_LOG) {\n \t\tint block_header_skip = 4 + header_off;\n@@ -203,7 +207,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\tuLongf src_len = block->len - block_header_skip;\n \t\t/* Log blocks specify the *uncompressed* size in their header.\n \t\t */\n-\t\tuint8_t *uncompressed = reftable_malloc(sz);\n+\t\tuncompressed = reftable_malloc(sz);\n \n \t\t/* Copy over the block header verbatim. It's not compressed. */\n \t\tmemcpy(uncompressed, block->data, block_header_skip);\n@@ -212,16 +216,19 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \t\tif (Z_OK !=\n \t\t uncompress2(uncompressed + block_header_skip, &dst_len,\n \t\t\t\tblock->data + block_header_skip, &src_len)) {\n-\t\t\treftable_free(uncompressed);\n-\t\t\treturn REFTABLE_ZLIB_ERROR;\n+\t\t\terr = REFTABLE_ZLIB_ERROR;\n+\t\t\tgoto done;\n \t\t}\n \n-\t\tif (dst_len + block_header_skip != sz)\n-\t\t\treturn REFTABLE_FORMAT_ERROR;\n+\t\tif (dst_len + block_header_skip != sz) {\n+\t\t\terr = REFTABLE_FORMAT_ERROR;\n+\t\t\tgoto done;\n+\t\t}\n \n \t\t/* We're done with the input data. */\n \t\treftable_block_done(block);\n \t\tblock->data = uncompressed;\n+\t\tuncompressed = NULL;\n \t\tblock->len = sz;\n \t\tblock->source = malloc_block_source();\n \t\tfull_block_size = src_len + block_header_skip;\n@@ -251,7 +258,9 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,\n \tbr->restart_count = restart_count;\n \tbr->restart_bytes = restart_bytes;\n \n-\treturn 0;\n+done:\n+\treftable_free(uncompressed);\n+\treturn err;\n }\n \n static uint32_t block_reader_restart_offset(struct block_reader *br, int i)\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 006709a645a..35781593a29 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -290,28 +290,33 @@ int reader_init_block_reader(struct reftable_reader *r, struct block_reader *br,\n \n \terr = reader_get_block(r, &block, next_off, guess_block_size);\n \tif (err < 0)\n-\t\treturn err;\n+\t\tgoto done;\n \n \tblock_size = extract_block_size(block.data, &block_typ, next_off,\n \t\t\t\t\tr->version);\n-\tif (block_size < 0)\n-\t\treturn block_size;\n-\n+\tif (block_size < 0) {\n+\t\terr = block_size;\n+\t\tgoto done;\n+\t}\n \tif (want_typ != BLOCK_TYPE_ANY && block_typ != want_typ) {\n-\t\treftable_block_done(&block);\n-\t\treturn 1;\n+\t\terr = 1;\n+\t\tgoto done;\n \t}\n \n \tif (block_size > guess_block_size) {\n \t\treftable_block_done(&block);\n \t\terr = reader_get_block(r, &block, next_off, block_size);\n \t\tif (err < 0) {\n-\t\t\treturn err;\n+\t\t\tgoto done;\n \t\t}\n \t}\n \n-\treturn block_reader_init(br, &block, header_off, r->block_size,\n-\t\t\t\t hash_size(r->hash_id));\n+\terr = block_reader_init(br, &block, header_off, r->block_size,\n+\t\t\t\thash_size(r->hash_id));\n+done:\n+\treftable_block_done(&block);\n+\n+\treturn err;\n }\n \n static int table_iter_next_block(struct table_iter *dest,\ndiff --git a/reftable/readwrite_test.c b/reftable/readwrite_test.c\nindex 70c7aedba2c..edb9cfc1930 100644\n--- a/reftable/readwrite_test.c\n+++ b/reftable/readwrite_test.c\n@@ -288,6 +288,71 @@ static void test_log_write_read(void)\n \treader_close(&rd);\n }\n \n+static void test_log_zlib_corruption(void)\n+{\n+\tstruct reftable_write_options opts = {\n+\t\t.block_size = 256,\n+\t};\n+\tstruct reftable_iterator it = { 0 };\n+\tstruct reftable_reader rd = { 0 };\n+\tstruct reftable_block_source source = { 0 };\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tstruct reftable_writer *w =\n+\t\treftable_new_writer(&strbuf_add_void, &buf, &opts);\n+\tconst struct reftable_stats *stats = NULL;\n+\tuint8_t hash1[GIT_SHA1_RAWSZ] = { 1 };\n+\tuint8_t hash2[GIT_SHA1_RAWSZ] = { 2 };\n+\tchar message[100] = { 0 };\n+\tint err, i, n;\n+\n+\tstruct reftable_log_record log = {\n+\t\t.refname = \"refname\",\n+\t\t.value_type = REFTABLE_LOG_UPDATE,\n+\t\t.value = {\n+\t\t\t.update = {\n+\t\t\t\t.new_hash = hash1,\n+\t\t\t\t.old_hash = hash2,\n+\t\t\t\t.name = \"My Name\",\n+\t\t\t\t.email = \"myname@invalid\",\n+\t\t\t\t.message = message,\n+\t\t\t},\n+\t\t},\n+\t};\n+\n+\tfor (i = 0; i < sizeof(message) - 1; i++)\n+\t\tmessage[i] = (uint8_t)(rand() % 64 + ' ');\n+\n+\treftable_writer_set_limits(w, 1, 1);\n+\n+\terr = reftable_writer_add_log(w, &log);\n+\tEXPECT_ERR(err);\n+\n+\tn = reftable_writer_close(w);\n+\tEXPECT(n == 0);\n+\n+\tstats = writer_stats(w);\n+\tEXPECT(stats->log_stats.blocks > 0);\n+\treftable_writer_free(w);\n+\tw = NULL;\n+\n+\t/* corrupt the data. */\n+\tbuf.buf[50] ^= 0x99;\n+\n+\tblock_source_from_strbuf(&source, &buf);\n+\n+\terr = init_reader(&rd, &source, \"file.log\");\n+\tEXPECT_ERR(err);\n+\n+\terr = reftable_reader_seek_log(&rd, &it, \"refname\");\n+\tEXPECT(err == REFTABLE_ZLIB_ERROR);\n+\n+\treftable_iterator_destroy(&it);\n+\n+\t/* cleanup. */\n+\tstrbuf_release(&buf);\n+\treader_close(&rd);\n+}\n+\n static void test_table_read_write_sequential(void)\n {\n \tchar **names;\n@@ -667,6 +732,7 @@ static void test_corrupt_table(void)\n \n int readwrite_test_main(int argc, const char *argv[])\n {\n+\tRUN_TEST(test_log_zlib_corruption);\n \tRUN_TEST(test_corrupt_table);\n \tRUN_TEST(test_corrupt_table_empty);\n \tRUN_TEST(test_log_write_read);\n-- \ngitgitgadget\n\n"},{"id":"446788","messageId":"f11a996438360c209f9da2db768e2be98c4413a9.1643051624.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v7.git.git.1643051624.gitgitgadget@gmail.com","subject":"[PATCH v7 03/16] reftable: fix resource leak blocksource.c","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-24T19:13:31Z","receivedAt":"2022-01-24T19:38:35Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis would be triggered in the unlikely event of fstat() failing on an\nopened file.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/blocksource.c | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/reftable/blocksource.c b/reftable/blocksource.c\nindex 0044eecd9aa..2605371c28d 100644\n--- a/reftable/blocksource.c\n+++ b/reftable/blocksource.c\n@@ -134,8 +134,10 @@ int reftable_block_source_from_file(struct reftable_block_source *bs,\n \t}\n \n \terr = fstat(fd, &st);\n-\tif (err < 0)\n-\t\treturn -1;\n+\tif (err < 0) {\n+\t\tclose(fd);\n+\t\treturn REFTABLE_IO_ERROR;\n+\t}\n \n \tp = reftable_calloc(sizeof(struct file_block_source));\n \tp->size = st.st_size;\n-- \ngitgitgadget\n\n"},{"id":"446789","messageId":"cc47ec7a50b44e6734a36e1fdb7c21e798efab7e.1643051624.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v7.git.git.1643051624.gitgitgadget@gmail.com","subject":"[PATCH v7 01/16] reftable: fix OOB stack write in print functions","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-24T19:13:29Z","receivedAt":"2022-01-24T19:38:44Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nRemove /* BUG */ comment, whose meaning has escaped my memory.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 6a5dac32dc6..1f236b47d1c 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -254,7 +254,7 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)\n void reftable_ref_record_print(struct reftable_ref_record *ref,\n \t\t\t uint32_t hash_id)\n {\n-\tchar hex[2 * GIT_SHA256_RAWSZ + 1] = { 0 }; /* BUG */\n+\tchar hex[GIT_MAX_HEXSZ + 1];\n \tprintf(\"ref{%s(%\" PRIu64 \") \", ref->refname, ref->update_index);\n \tswitch (ref->value_type) {\n \tcase REFTABLE_REF_SYMREF:\n@@ -586,7 +586,7 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {\n void reftable_log_record_print(struct reftable_log_record *log,\n \t\t\t uint32_t hash_id)\n {\n-\tchar hex[GIT_SHA256_RAWSZ + 1] = { 0 };\n+\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 };\n \n \tswitch (log->value_type) {\n \tcase REFTABLE_LOG_DELETION:\n-- \ngitgitgadget\n\n"},{"id":"446790","messageId":"pull.1152.v7.git.git.1643051624.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v6.git.git.1642691534.gitgitgadget@gmail.com","subject":"[PATCH v7 00/16] Reftable coverity fixes","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-24T19:13:28Z","receivedAt":"2022-01-24T19:38:47Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"This series was targeted to 'master'.\n\nThis addresses some complaints from Coverity that Peff reported.\n\nv7:\n\n * address avarab's comments.\n\nHan-Wen Nienhuys (16):\n reftable: fix OOB stack write in print functions\n reftable: fix resource leak in block.c error path\n reftable: fix resource leak blocksource.c\n reftable: check reftable_stack_auto_compact() return value\n reftable: ignore remove() return value in stack_test.c\n reftable: fix resource warning\n reftable: all xxx_free() functions accept NULL arguments\n reftable: order unittests by complexity\n reftable: drop stray printf in readwrite_test\n reftable: handle null refnames in reftable_ref_record_equal\n reftable: make reftable-record.h function signatures const correct\n reftable: implement record equality generically\n reftable: remove outdated file reftable.c\n reftable: make reftable_record a tagged union\n reftable: add print functions to the record types\n reftable: rename typ to type\n\n reftable/block.c | 35 ++--\n reftable/block_test.c | 22 +--\n reftable/blocksource.c | 6 +-\n reftable/generic.c | 39 ++--\n reftable/iter.c | 4 +-\n reftable/merged.c | 35 ++--\n reftable/merged.h | 2 +-\n reftable/pq.c | 2 +-\n reftable/pq_test.c | 29 ++-\n reftable/reader.c | 167 +++++++++--------\n reftable/readwrite_test.c | 67 ++++++-\n reftable/record.c | 367 +++++++++++++++++++++++--------------\n reftable/record.h | 49 ++---\n reftable/record_test.c | 193 +++++++++----------\n reftable/reftable-record.h | 14 +-\n reftable/reftable.c | 115 ------------\n reftable/stack.c | 10 +-\n reftable/stack_test.c | 3 +-\n reftable/writer.c | 72 ++++----\n t/helper/test-reftable.c | 9 +-\n 20 files changed, 666 insertions(+), 574 deletions(-)\n delete mode 100644 reftable/reftable.c\n\n\nbase-commit: 50b2d72e110cad39ecaf2322bfdf1b60cd13dd96\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1152%2Fhanwen%2Freftable-coverity-v7\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1152/hanwen/reftable-coverity-v7\nPull-Request: https://github.com/git/git/pull/1152\n\nRange-diff vs v6:\n\n 1: 9218bd59b25 ! 1: cc47ec7a50b reftable: fix OOB stack write in print functions\n @@ Metadata\n ## Commit message ##\n reftable: fix OOB stack write in print functions\n \n + Remove /* BUG */ comment, whose meaning has escaped my memory.\n +\n Signed-off-by: Han-Wen Nienhuys \n \n ## reftable/record.c ##\n @@ reftable/record.c: static void hex_format(char *dest, uint8_t *src, int hash_siz\n \t\t\t uint32_t hash_id)\n {\n -\tchar hex[2 * GIT_SHA256_RAWSZ + 1] = { 0 }; /* BUG */\n -+\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */\n ++\tchar hex[GIT_MAX_HEXSZ + 1];\n \tprintf(\"ref{%s(%\" PRIu64 \") \", ref->refname, ref->update_index);\n \tswitch (ref->value_type) {\n \tcase REFTABLE_REF_SYMREF:\n 2: 315ce62e710 ! 2: 1c5441d2f7e reftable: fix resource leak in block.c error path\n @@ Commit message\n \n ## reftable/block.c ##\n @@ reftable/block.c: int block_reader_init(struct block_reader *br, struct reftable_block *block,\n - \tuint32_t full_block_size = table_block_size;\n \tuint8_t typ = block->data[header_off];\n \tuint32_t sz = get_be24(block->data + header_off + 1);\n --\n + \n +\tint err = 0;\n \tuint16_t restart_count = 0;\n \tuint32_t restart_start = 0;\n 3: 7cf65e141cf = 3: f11a9964383 reftable: fix resource leak blocksource.c\n 4: 8740ba92981 ! 4: b0bffb6e29d reftable: check reftable_stack_auto_compact() return value\n @@ Metadata\n ## Commit message ##\n reftable: check reftable_stack_auto_compact() return value\n \n - Fixes a problem detected by Coverity.\n + If compaction consistently errors out, we would accumulate reftables,\n + and the follow-on EXPECT would start failing, but catching errors\n + early helps debugging.\n +\n + Spotted by Coverity.\n \n Signed-off-by: Han-Wen Nienhuys \n \n 5: 08be6d90a48 ! 5: e3d575ae2b4 reftable: ignore remove() return value in stack_test.c\n @@ reftable/stack_test.c: static void test_read_file(void)\n \t}\n \tfree_names(names);\n -\tremove(fn);\n -+\t(void) remove(fn);\n ++\tEXPECT(unlink(fn) == 0);\n }\n \n static void test_parse_names(void)\n 6: 630f67bdb51 = 6: a21d934f37d reftable: fix resource warning\n 7: afee67fb60d = 7: b40d91c50a2 reftable: all xxx_free() functions accept NULL arguments\n 8: 6f4f24c17ad = 8: 7a1f32a2ff0 reftable: order unittests by complexity\n 9: 0abaab1d7c4 = 9: a118cd40975 reftable: drop stray printf in readwrite_test\n 10: 004b7a736bf = 10: 1d5cfbbc47a reftable: handle null refnames in reftable_ref_record_equal\n 11: 951efcebcd2 ! 11: 90c72a2df0c reftable: make reftable-record.h function signatures const correct\n @@ reftable/record.c: static void hex_format(char *dest, uint8_t *src, int hash_siz\n +void reftable_ref_record_print(const struct reftable_ref_record *ref,\n \t\t\t uint32_t hash_id)\n {\n - \tchar hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */\n + \tchar hex[GIT_MAX_HEXSZ + 1];\n @@ reftable/record.c: static int zero_hash_eq(uint8_t *a, uint8_t *b, int sz)\n \treturn !memcmp(a, b, sz);\n }\n 12: 6385e449ba7 ! 12: a83a9cf59b7 reftable: implement record equality generically\n @@ reftable/record.c: static int reftable_ref_record_is_deletion_void(const void *p\n \t\t(const struct reftable_ref_record *)p);\n }\n \n -+\n +static int reftable_ref_record_equal_void(const void *a,\n +\t\t\t\t\t const void *b, int hash_size)\n +{\n @@ reftable/record.c: static int not_a_deletion(const void *p)\n +\t || ra->offset_len != rb->offset_len)\n +\t\treturn 0;\n +\n -+\tif (ra->hash_prefix_len &&\n -+\t memcmp(ra->hash_prefix, rb->hash_prefix, ra->hash_prefix_len))\n ++\tif (memcmp(ra->hash_prefix, rb->hash_prefix, ra->hash_prefix_len))\n +\t\treturn 0;\n -+\tif (ra->offset_len &&\n -+\t memcmp(ra->offsets, rb->offsets, ra->offset_len * sizeof(uint64_t)))\n ++\tif (memcmp(ra->offsets, rb->offsets, ra->offset_len * sizeof(uint64_t)))\n +\t\treturn 0;\n +\n +\treturn 1;\n 13: 69c4a554882 = 13: 7e23d14d1af reftable: remove outdated file reftable.c\n 14: 54bcac325bd ! 14: 047bbb75fcb reftable: make reftable_record a tagged union\n @@ reftable/generic.c: https://developers.google.com/open-source/licenses/bsd\n {\n -\tstruct reftable_ref_record ref = {\n -\t\t.refname = (char *)name,\n --\t};\n ++\tstruct reftable_record rec = {\n ++\t\t.type = BLOCK_TYPE_REF,\n ++\t\t.u.ref = { .refname = (char *)name, }\n + \t};\n -\tstruct reftable_record rec = { NULL };\n -\treftable_record_from_ref(&rec, &ref);\n -+\tstruct reftable_record rec = { .type = BLOCK_TYPE_REF,\n -+\t\t\t\t .u.ref = {\n -+\t\t\t\t\t .refname = (char *)name,\n -+\t\t\t\t } };\n \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n }\n \n @@ reftable/generic.c: https://developers.google.com/open-source/licenses/bsd\n -\tstruct reftable_log_record log = {\n -\t\t.refname = (char *)name,\n -\t\t.update_index = ~((uint64_t)0),\n --\t};\n ++\tstruct reftable_record rec = {\n ++\t\t.type = BLOCK_TYPE_LOG,\n ++\t\t.u.log = {\n ++\t\t\t.refname = (char *)name,\n ++\t\t\t.update_index = ~((uint64_t)0),\n ++\t\t}\n + \t};\n -\tstruct reftable_record rec = { NULL };\n -\treftable_record_from_log(&rec, &log);\n -+\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n -+\t\t\t\t .u.log = {\n -+\t\t\t\t\t .refname = (char *)name,\n -+\t\t\t\t\t .update_index = ~((uint64_t)0),\n -+\t\t\t\t } };\n \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n }\n \n @@ reftable/merged.c: int reftable_merged_table_seek_ref(struct reftable_merged_tab\n -\t\t.refname = (char *)name,\n +\tstruct reftable_record rec = {\n +\t\t.type = BLOCK_TYPE_REF,\n -+\t\t.u.ref = {\n -+\t\t\t.refname = (char *)name,\n -+\t\t},\n ++\t\t.u.ref = { .refname = (char *)name, },\n \t};\n -\tstruct reftable_record rec = { NULL };\n -\treftable_record_from_ref(&rec, &ref);\n @@ reftable/merged.c: int reftable_merged_table_seek_log_at(struct reftable_merged_\n -\tstruct reftable_log_record log = {\n -\t\t.refname = (char *)name,\n -\t\t.update_index = update_index,\n --\t};\n ++\tstruct reftable_record rec = {\n ++\t\t.type = BLOCK_TYPE_LOG,\n ++\t\t.u.log = {\n ++\t\t\t.refname = (char *)name,\n ++\t\t\t.update_index = update_index,\n ++\t\t}\n + \t};\n -\tstruct reftable_record rec = { NULL };\n -\treftable_record_from_log(&rec, &log);\n -+\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n -+\t\t\t\t .u.log = {\n -+\t\t\t\t\t .refname = (char *)name,\n -+\t\t\t\t\t .update_index = update_index,\n -+\t\t\t\t } };\n \treturn merged_table_seek_record(mt, it, &rec);\n }\n \n \n ## reftable/pq.c ##\n -@@ reftable/pq.c: struct pq_entry merged_iter_pqueue_remove(struct merged_iter_pqueue *pq)\n - void merged_iter_pqueue_add(struct merged_iter_pqueue *pq, struct pq_entry e)\n - {\n - \tint i = 0;\n -+\n - \tif (pq->len == pq->cap) {\n - \t\tpq->cap = 2 * pq->cap + 1;\n - \t\tpq->heap = reftable_realloc(pq->heap,\n @@ reftable/pq.c: void merged_iter_pqueue_release(struct merged_iter_pqueue *pq)\n {\n \tint i = 0;\n @@ reftable/pq_test.c: static void test_pq(void)\n -\n -\t\treftable_record_as_ref(&rec)->refname = names[i];\n -\t\te.rec = rec;\n -+\t\tstruct pq_entry e = { .rec = { .type = BLOCK_TYPE_REF,\n -+\t\t\t\t\t .u.ref = {\n -+\t\t\t\t\t\t .refname = names[i],\n -+\t\t\t\t\t } } };\n ++\t\tstruct pq_entry e = {\n ++\t\t\t.rec = {\n ++\t\t\t\t.type = BLOCK_TYPE_REF,\n ++\t\t\t\t.u.ref = { .refname = names[i], }\n ++\t\t\t}\n ++\t\t};\n \t\tmerged_iter_pqueue_add(&pq, e);\n \t\tmerged_iter_pqueue_check(pq);\n \t\ti = (i * 7) % N;\n @@ reftable/pq_test.c: static void test_pq(void)\n -\t\tlast = ref->refname;\n -\t\tref->refname = NULL;\n -\t\treftable_free(ref);\n -+\t\t// this is names[i], so don't dealloc.\n ++\t\t/* this is names[i], so don't dealloc. */\n +\t\tlast = rec->u.ref.refname;\n +\t\trec->u.ref.refname = NULL;\n +\t\treftable_record_release(rec);\n @@ reftable/reader.c: static int reader_seek_indexed(struct reftable_reader *r,\n -\treftable_record_key(rec, &want_index.last_key);\n -\treftable_record_from_index(&want_index_rec, &want_index);\n -\treftable_record_from_index(&index_result_rec, &index_result);\n --\n +\treftable_record_key(rec, &want_index.u.idx.last_key);\n + \n \terr = reader_start(r, &index_iter, reftable_record_type(rec), 1);\n \tif (err < 0)\n \t\tgoto done;\n @@ reftable/reader.c: static int reader_seek(struct reftable_reader *r, struct reft\n -\t\t.refname = (char *)name,\n +\tstruct reftable_record rec = {\n +\t\t.type = BLOCK_TYPE_REF,\n -+\t\t.u.ref = {\n -+\t\t\t.refname = (char *)name,\n -+\t\t},\n ++\t\t.u.ref = { .refname = (char *)name, },\n \t};\n -\tstruct reftable_record rec = { NULL };\n -\treftable_record_from_ref(&rec, &ref);\n @@ reftable/reader.c: int reftable_reader_seek_log_at(struct reftable_reader *r,\n -\tstruct reftable_log_record log = {\n -\t\t.refname = (char *)name,\n -\t\t.update_index = update_index,\n --\t};\n ++\tstruct reftable_record rec = {\n ++\t\t.type = BLOCK_TYPE_LOG,\n ++\t\t.u.log = {\n ++\t\t\t.refname = (char *)name,\n ++\t\t\t.update_index = update_index,\n ++\t\t}\n + \t};\n -\tstruct reftable_record rec = { NULL };\n -\treftable_record_from_log(&rec, &log);\n -+\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG,\n -+\t\t\t\t .u.log = {\n -+\t\t\t\t\t .refname = (char *)name,\n -+\t\t\t\t\t .update_index = update_index,\n -+\t\t\t\t } };\n \treturn reader_seek(r, it, &rec);\n }\n \n @@ reftable/record.c: https://developers.google.com/open-source/licenses/bsd\n \n +static struct reftable_record_vtable *\n +reftable_record_vtable(struct reftable_record *rec);\n -+static void *reftable_record_data(struct reftable_record *rec);\n ++\n ++static void *reftable_record_data(struct reftable_record *rec)\n ++{\n ++\tswitch (rec->type) {\n ++\tcase BLOCK_TYPE_REF:\n ++\t\treturn &rec->u.ref;\n ++\tcase BLOCK_TYPE_LOG:\n ++\t\treturn &rec->u.log;\n ++\tcase BLOCK_TYPE_INDEX:\n ++\t\treturn &rec->u.idx;\n ++\tcase BLOCK_TYPE_OBJ:\n ++\t\treturn &rec->u.obj;\n ++\t}\n ++\tabort();\n ++}\n +\n int get_var_int(uint64_t *dest, struct string_view *in)\n {\n @@ reftable/record.c: static void reftable_obj_record_copy_from(void *rec, const vo\n \treftable_obj_record_release(obj);\n -\t*obj = *src;\n -\tobj->hash_prefix = reftable_malloc(obj->hash_prefix_len);\n --\tmemcpy(obj->hash_prefix, src->hash_prefix, obj->hash_prefix_len);\n +\tobj->hash_prefix = reftable_malloc(src->hash_prefix_len);\n +\tobj->hash_prefix_len = src->hash_prefix_len;\n -+\tif (src->hash_prefix_len)\n -+\t\tmemcpy(obj->hash_prefix, src->hash_prefix, obj->hash_prefix_len);\n + \tmemcpy(obj->hash_prefix, src->hash_prefix, obj->hash_prefix_len);\n \n -\tobj->offsets = reftable_malloc(obj->offset_len * sizeof(uint64_t));\n -\tCOPY_ARRAY(obj->offsets, src->offsets, obj->offset_len);\n @@ reftable/record.c: static struct reftable_record_vtable reftable_log_record_vtab\n static void reftable_index_record_key(const void *r, struct strbuf *dest)\n {\n \tconst struct reftable_index_record *rec = r;\n +@@ reftable/record.c: static void reftable_index_record_release(void *rec)\n + {\n + \tstruct reftable_index_record *idx = rec;\n + \tstrbuf_release(&idx->last_key);\n ++\tidx->offset = 0;\n + }\n + \n + static uint8_t reftable_index_record_val_type(const void *rec)\n @@ reftable/record.c: static struct reftable_record_vtable reftable_index_record_vtable = {\n \n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)\n @@ reftable/record.c: void string_view_consume(struct string_view *s, int n)\n \ts->len -= n;\n }\n +\n -+static void *reftable_record_data(struct reftable_record *rec)\n -+{\n -+\tswitch (rec->type) {\n -+\tcase BLOCK_TYPE_REF:\n -+\t\treturn &rec->u.ref;\n -+\tcase BLOCK_TYPE_LOG:\n -+\t\treturn &rec->u.log;\n -+\tcase BLOCK_TYPE_INDEX:\n -+\t\treturn &rec->u.idx;\n -+\tcase BLOCK_TYPE_OBJ:\n -+\t\treturn &rec->u.obj;\n -+\t}\n -+\tabort();\n -+}\n -+\n +static struct reftable_record_vtable *\n +reftable_record_vtable(struct reftable_record *rec)\n +{\n @@ reftable/record.c: void string_view_consume(struct string_view *s, int n)\n +\tabort();\n +}\n +\n -+struct reftable_record reftable_new_record(uint8_t typ)\n ++struct reftable_record reftable_new_record(uint8_t type)\n +{\n +\tstruct reftable_record clean = {\n -+\t\t.type = typ,\n ++\t\t.type = type,\n +\t};\n +\n +\t/* the following is involved, but the naive solution (just return\n +\t * `clean` as is, except for BLOCK_TYPE_INDEX), returns a garbage\n +\t * clean.u.obj.offsets pointer on Windows VS CI. Go figure.\n +\t */\n -+\tswitch (typ) {\n ++\tswitch (type) {\n +\tcase BLOCK_TYPE_OBJ:\n +\t{\n +\t\tstruct reftable_obj_record obj = { 0 };\n @@ reftable/record_test.c\n -\tstruct reftable_record copy =\n -\t\treftable_new_record(reftable_record_type(rec));\n +\tstruct reftable_record copy = { 0 };\n -+\tuint8_t typ;\n -+\n -+\ttyp = reftable_record_type(rec);\n -+\tcopy = reftable_new_record(typ);\n ++\tuint8_t type = reftable_record_type(rec);\n ++\tcopy = reftable_new_record(type);\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \t/* do it twice to catch memory leaks */\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n @@ reftable/record_test.c: static void test_reftable_ref_record_roundtrip(void)\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n \t\t\t.buf = buffer,\n - \t\t\t.len = sizeof(buffer),\n - \t\t};\n --\n +@@ reftable/record_test.c: static void test_reftable_ref_record_roundtrip(void)\n + \n \t\tint n, m;\n \n -\t\tin.value_type = i;\n @@ reftable/record_test.c: static void test_reftable_ref_record_roundtrip(void)\n -\t\t\tset_hash(in.value.val2.value, 1);\n -\t\t\tin.value.val2.target_value =\n +\t\t\tin.u.ref.value.val2.value =\n - \t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n --\t\t\tset_hash(in.value.val2.target_value, 2);\n ++\t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n +\t\t\tset_hash(in.u.ref.value.val2.value, 1);\n +\t\t\tin.u.ref.value.val2.target_value =\n -+\t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n + \t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n +-\t\t\tset_hash(in.value.val2.target_value, 2);\n +\t\t\tset_hash(in.u.ref.value.val2.target_value, 2);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_SYMREF:\n @@ reftable/writer.c: int reftable_writer_add_refs(struct reftable_writer *w,\n \t block_writer_type(w->block_writer) == BLOCK_TYPE_REF) {\n \t\tint err = writer_finish_public_section(w);\n @@ reftable/writer.c: static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n - \n \tw->next -= w->pending_padding;\n \tw->pending_padding = 0;\n --\n + \n -\treftable_record_from_log(&rec, log);\n \treturn writer_add_record(w, &rec);\n }\n @@ reftable/writer.c: static void write_object_record(void *void_arg, void *key)\n -\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n -\t\t.offsets = entry->offsets,\n -\t\t.offset_len = entry->offset_len,\n --\t};\n ++\tstruct reftable_record rec = {\n ++\t\t.type = BLOCK_TYPE_OBJ,\n ++\t\t.u.obj = {\n ++\t\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n ++\t\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n ++\t\t\t.offsets = entry->offsets,\n ++\t\t\t.offset_len = entry->offset_len,\n ++\t\t}\n + \t};\n -\tstruct reftable_record rec = { NULL };\n -+\tstruct reftable_record\n -+\t\trec = { .type = BLOCK_TYPE_OBJ,\n -+\t\t\t.u.obj = {\n -+\t\t\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n -+\t\t\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n -+\t\t\t\t.offsets = entry->offsets,\n -+\t\t\t\t.offset_len = entry->offset_len,\n -+\t\t\t} };\n \tif (arg->err < 0)\n \t\tgoto done;\n \n @@ reftable/writer.c: static void write_object_record(void *void_arg, void *key)\n \tif (arg->err == 0)\n \t\tgoto done;\n -\tobj_rec.offset_len = 0;\n -+\n +\trec.u.obj.offset_len = 0;\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \n 15: 82f140cab5c ! 15: 1e8785bbc26 reftable: add print functions to the record types\n @@ reftable/record.c: static void hex_format(char *dest, uint8_t *src, int hash_siz\n +static void reftable_ref_record_print_sz(const struct reftable_ref_record *ref,\n +\t\t\t\t\t int hash_size)\n {\n - \tchar hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */\n + \tchar hex[GIT_MAX_HEXSZ + 1];\n \tprintf(\"ref{%s(%\" PRIu64 \") \", ref->refname, ref->update_index);\n @@ reftable/record.c: void reftable_ref_record_print(const struct reftable_ref_record *ref,\n \t\tprintf(\"=> %s\", ref->value.symref);\n @@ reftable/record.c: static int reftable_ref_record_equal_void(const void *a,\n +static void reftable_ref_record_print_void(const void *rec,\n +\t\t\t\t\t int hash_size)\n +{\n -+\treftable_ref_record_print_sz((struct reftable_ref_record *) rec, hash_size);\n ++\treftable_ref_record_print_sz(\n ++\t\t(struct reftable_ref_record *) rec, hash_size);\n +}\n +\n static struct reftable_record_vtable reftable_ref_record_vtable = {\n @@ reftable/record.c: static void reftable_obj_record_release(void *rec)\n +\tint i;\n +\n +\tfor (i = 0; i < obj->offset_len; i++)\n -+\t\tstrbuf_addf(&offset_str, \"%\" PRIu64 \" \", obj->offsets[i]);\n ++\t\tstrbuf_addf(&offset_str, \"%\"PRIu64 \" \", obj->offsets[i]);\n +\thex_format(hex, obj->hash_prefix, obj->hash_prefix_len);\n +\tprintf(\"prefix %s (len %d), offsets [%s]\\n\",\n +\t hex, obj->hash_prefix_len, offset_str.buf);\n @@ reftable/record.c: static struct reftable_record_vtable reftable_obj_record_vtab\n \tswitch (log->value_type) {\n \tcase REFTABLE_LOG_DELETION:\n -\t\tprintf(\"log{%s(%\" PRIu64 \") delete\", log->refname,\n -+\t\tprintf(\"log{%s(%\" PRIu64 \") delete\\n\", log->refname,\n ++\t\tprintf(\"log{%s(%\"PRIu64 \") delete\\n\", log->refname,\n \t\t log->update_index);\n \t\tbreak;\n \tcase REFTABLE_LOG_UPDATE:\n - \t\tprintf(\"log{%s(%\" PRIu64 \") %s <%s> %\" PRIu64 \" %04d\\n\",\n +-\t\tprintf(\"log{%s(%\" PRIu64 \") %s <%s> %\" PRIu64 \" %04d\\n\",\n -\t\t log->refname, log->update_index, log->value.update.name,\n -\t\t log->value.update.email, log->value.update.time,\n ++\t\tprintf(\"log{%s(%\"PRIu64 \") %s <%s> %\" PRIu64 \" %04d\\n\",\n +\t\t log->refname, log->update_index,\n +\t\t log->value.update.name ? log->value.update.name : \"\",\n +\t\t log->value.update.email ? log->value.update.email : \"\",\n @@ reftable/record.c: static struct reftable_record_vtable reftable_index_record_vt\n };\n \n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)\n -@@ reftable/record.c: struct reftable_record reftable_new_record(uint8_t typ)\n +@@ reftable/record.c: struct reftable_record reftable_new_record(uint8_t type)\n \t}\n \treturn clean;\n }\n @@ reftable/record_test.c: static void test_reftable_log_record_equal(void)\n {\n \tint i;\n -\tstruct reftable_log_record in[2] = {\n -+\n +\tstruct reftable_log_record in[] = {\n \t\t{\n \t\t\t.refname = xstrdup(\"refs/heads/master\"),\n @@ reftable/record_test.c: static void test_reftable_log_record_roundtrip(void)\n \t\t\t.value_type = REFTABLE_LOG_DELETION,\n +\t\t},\n +\t\t{\n ++\t\t\t/* make sure we handle optional fields correctly. */\n +\t\t\t.refname = xstrdup(\"branch\"),\n +\t\t\t.update_index = 33,\n +\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n @@ reftable/record_test.c: static void test_reftable_log_record_roundtrip(void)\n +\t\t\t\t.update = {\n +\t\t\t\t\t.old_hash = reftable_malloc(GIT_SHA1_RAWSZ),\n +\t\t\t\t\t.new_hash = reftable_malloc(GIT_SHA1_RAWSZ),\n -+\t\t\t\t\t/* rest of fields left empty. */\n +\t\t\t\t},\n +\t\t\t},\n \t\t}\n -: ----------- > 16: 556fc11080a reftable: rename typ to type\n\n-- \ngitgitgadget\n"},{"id":"446791","messageId":"e3d575ae2b474c1e4510dd9b037bd61f58d06547.1643051624.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v7.git.git.1643051624.gitgitgadget@gmail.com","subject":"[PATCH v7 05/16] reftable: ignore remove() return value in stack_test.c","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-24T19:13:33Z","receivedAt":"2022-01-24T19:38:58Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nIf the cleanup fails, there is nothing we can do.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack_test.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/reftable/stack_test.c b/reftable/stack_test.c\nindex e84f50d27ff..f0be0be9fee 100644\n--- a/reftable/stack_test.c\n+++ b/reftable/stack_test.c\n@@ -90,7 +90,7 @@ static void test_read_file(void)\n \t\tEXPECT(0 == strcmp(want[i], names[i]));\n \t}\n \tfree_names(names);\n-\tremove(fn);\n+\tEXPECT(unlink(fn) == 0);\n }\n \n static void test_parse_names(void)\n-- \ngitgitgadget\n\n"},{"id":"446792","messageId":"a21d934f37d9951f51ce48bd14e1355f72ba4396.1643051624.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v7.git.git.1643051624.gitgitgadget@gmail.com","subject":"[PATCH v7 06/16] reftable: fix resource warning","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-24T19:13:34Z","receivedAt":"2022-01-24T19:39:05Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis would trigger in the unlikely event that we are compacting, and\nthe next available file handle is 0.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/stack.c | 10 +++++-----\n 1 file changed, 5 insertions(+), 5 deletions(-)\n\ndiff --git a/reftable/stack.c b/reftable/stack.c\nindex 56bf5f2d84a..ddbdf1b9c8b 100644\n--- a/reftable/stack.c\n+++ b/reftable/stack.c\n@@ -889,7 +889,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \tstruct strbuf new_table_path = STRBUF_INIT;\n \tint err = 0;\n \tint have_lock = 0;\n-\tint lock_file_fd = 0;\n+\tint lock_file_fd = -1;\n \tint compact_count = last - first + 1;\n \tchar **listp = NULL;\n \tchar **delete_on_success =\n@@ -923,7 +923,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \t}\n \t/* Don't want to write to the lock for now. */\n \tclose(lock_file_fd);\n-\tlock_file_fd = 0;\n+\tlock_file_fd = -1;\n \n \thave_lock = 1;\n \terr = stack_uptodate(st);\n@@ -1031,7 +1031,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,\n \t\tgoto done;\n \t}\n \terr = close(lock_file_fd);\n-\tlock_file_fd = 0;\n+\tlock_file_fd = -1;\n \tif (err < 0) {\n \t\terr = REFTABLE_IO_ERROR;\n \t\tunlink(new_table_path.buf);\n@@ -1068,9 +1068,9 @@ done:\n \t\tlistp++;\n \t}\n \tfree_names(subtable_locks);\n-\tif (lock_file_fd > 0) {\n+\tif (lock_file_fd >= 0) {\n \t\tclose(lock_file_fd);\n-\t\tlock_file_fd = 0;\n+\t\tlock_file_fd = -1;\n \t}\n \tif (have_lock) {\n \t\tunlink(lock_file_name.buf);\n-- \ngitgitgadget\n\n"},{"id":"446793","messageId":"b40d91c50a263e0e816a3b9826541a8bade267fc.1643051624.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v7.git.git.1643051624.gitgitgadget@gmail.com","subject":"[PATCH v7 07/16] reftable: all xxx_free() functions accept NULL arguments","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-24T19:13:35Z","receivedAt":"2022-01-24T19:39:06Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis fixes NULL derefs in error paths. Spotted by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/reader.c | 2 ++\n reftable/writer.c | 2 ++\n 2 files changed, 4 insertions(+)\n\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 35781593a29..272378ed1d5 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -646,6 +646,8 @@ int reftable_new_reader(struct reftable_reader **p,\n \n void reftable_reader_free(struct reftable_reader *r)\n {\n+\tif (!r)\n+\t\treturn;\n \treader_close(r);\n \treftable_free(r);\n }\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex 35c8649c9b7..83a23daf60b 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -150,6 +150,8 @@ void reftable_writer_set_limits(struct reftable_writer *w, uint64_t min,\n \n void reftable_writer_free(struct reftable_writer *w)\n {\n+\tif (!w)\n+\t\treturn;\n \treftable_free(w->block);\n \treftable_free(w);\n }\n-- \ngitgitgadget\n\n"},{"id":"446794","messageId":"7a1f32a2ff0926179066f09659f990d4a3f99566.1643051624.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v7.git.git.1643051624.gitgitgadget@gmail.com","subject":"[PATCH v7 08/16] reftable: order unittests by complexity","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-24T19:13:36Z","receivedAt":"2022-01-24T19:39:08Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis is a more practical ordering when working on refactorings of the\nreftable code.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n t/helper/test-reftable.c | 9 +++++----\n 1 file changed, 5 insertions(+), 4 deletions(-)\n\ndiff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c\nindex 26b03d7b789..1f0a28cbb64 100644\n--- a/t/helper/test-reftable.c\n+++ b/t/helper/test-reftable.c\n@@ -3,15 +3,16 @@\n \n int cmd__reftable(int argc, const char **argv)\n {\n+\t/* test from simple to complex. */\n \tbasics_test_main(argc, argv);\n+\trecord_test_main(argc, argv);\n \tblock_test_main(argc, argv);\n-\tmerged_test_main(argc, argv);\n+\ttree_test_main(argc, argv);\n \tpq_test_main(argc, argv);\n-\trecord_test_main(argc, argv);\n-\trefname_test_main(argc, argv);\n \treadwrite_test_main(argc, argv);\n+\tmerged_test_main(argc, argv);\n \tstack_test_main(argc, argv);\n-\ttree_test_main(argc, argv);\n+\trefname_test_main(argc, argv);\n \treturn 0;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"446795","messageId":"1d5cfbbc47a36e1f94d8432900da064a15c61b2d.1643051624.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v7.git.git.1643051624.gitgitgadget@gmail.com","subject":"[PATCH v7 10/16] reftable: handle null refnames in reftable_ref_record_equal","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-24T19:13:38Z","receivedAt":"2022-01-24T19:39:15Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSpotted by Coverity.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 1f236b47d1c..53d83e4a41a 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -1154,9 +1154,11 @@ int reftable_ref_record_equal(struct reftable_ref_record *a,\n \t\t\t struct reftable_ref_record *b, int hash_size)\n {\n \tassert(hash_size > 0);\n-\tif (!(0 == strcmp(a->refname, b->refname) &&\n-\t a->update_index == b->update_index &&\n-\t a->value_type == b->value_type))\n+\tif (!null_streq(a->refname, b->refname))\n+\t\treturn 0;\n+\n+\tif (a->update_index != b->update_index ||\n+\t a->value_type != b->value_type)\n \t\treturn 0;\n \n \tswitch (a->value_type) {\n-- \ngitgitgadget\n\n"},{"id":"446796","messageId":"90c72a2df0c40e5b4f1ad925a73cf0bed334348d.1643051624.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v7.git.git.1643051624.gitgitgadget@gmail.com","subject":"[PATCH v7 11/16] reftable: make reftable-record.h function signatures const correct","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-24T19:13:39Z","receivedAt":"2022-01-24T19:39:16Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 14 +++++++-------\n reftable/reftable-record.h | 14 +++++++-------\n 2 files changed, 14 insertions(+), 14 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 53d83e4a41a..1ae344c745b 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -72,7 +72,7 @@ int reftable_is_block_type(uint8_t typ)\n \treturn 0;\n }\n \n-uint8_t *reftable_ref_record_val1(struct reftable_ref_record *rec)\n+uint8_t *reftable_ref_record_val1(const struct reftable_ref_record *rec)\n {\n \tswitch (rec->value_type) {\n \tcase REFTABLE_REF_VAL1:\n@@ -84,7 +84,7 @@ uint8_t *reftable_ref_record_val1(struct reftable_ref_record *rec)\n \t}\n }\n \n-uint8_t *reftable_ref_record_val2(struct reftable_ref_record *rec)\n+uint8_t *reftable_ref_record_val2(const struct reftable_ref_record *rec)\n {\n \tswitch (rec->value_type) {\n \tcase REFTABLE_REF_VAL2:\n@@ -251,7 +251,7 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)\n \t}\n }\n \n-void reftable_ref_record_print(struct reftable_ref_record *ref,\n+void reftable_ref_record_print(const struct reftable_ref_record *ref,\n \t\t\t uint32_t hash_id)\n {\n \tchar hex[GIT_MAX_HEXSZ + 1];\n@@ -881,8 +881,8 @@ static int zero_hash_eq(uint8_t *a, uint8_t *b, int sz)\n \treturn !memcmp(a, b, sz);\n }\n \n-int reftable_log_record_equal(struct reftable_log_record *a,\n-\t\t\t struct reftable_log_record *b, int hash_size)\n+int reftable_log_record_equal(const struct reftable_log_record *a,\n+\t\t\t const struct reftable_log_record *b, int hash_size)\n {\n \tif (!(null_streq(a->refname, b->refname) &&\n \t a->update_index == b->update_index &&\n@@ -1150,8 +1150,8 @@ static int hash_equal(uint8_t *a, uint8_t *b, int hash_size)\n \treturn a == b;\n }\n \n-int reftable_ref_record_equal(struct reftable_ref_record *a,\n-\t\t\t struct reftable_ref_record *b, int hash_size)\n+int reftable_ref_record_equal(const struct reftable_ref_record *a,\n+\t\t\t const struct reftable_ref_record *b, int hash_size)\n {\n \tassert(hash_size > 0);\n \tif (!null_streq(a->refname, b->refname))\ndiff --git a/reftable/reftable-record.h b/reftable/reftable-record.h\nindex 5370d2288c7..67104f8fbfe 100644\n--- a/reftable/reftable-record.h\n+++ b/reftable/reftable-record.h\n@@ -49,25 +49,25 @@ struct reftable_ref_record {\n \n /* Returns the first hash, or NULL if `rec` is not of type\n * REFTABLE_REF_VAL1 or REFTABLE_REF_VAL2. */\n-uint8_t *reftable_ref_record_val1(struct reftable_ref_record *rec);\n+uint8_t *reftable_ref_record_val1(const struct reftable_ref_record *rec);\n \n /* Returns the second hash, or NULL if `rec` is not of type\n * REFTABLE_REF_VAL2. */\n-uint8_t *reftable_ref_record_val2(struct reftable_ref_record *rec);\n+uint8_t *reftable_ref_record_val2(const struct reftable_ref_record *rec);\n \n /* returns whether 'ref' represents a deletion */\n int reftable_ref_record_is_deletion(const struct reftable_ref_record *ref);\n \n /* prints a reftable_ref_record onto stdout. Useful for debugging. */\n-void reftable_ref_record_print(struct reftable_ref_record *ref,\n+void reftable_ref_record_print(const struct reftable_ref_record *ref,\n \t\t\t uint32_t hash_id);\n \n /* frees and nulls all pointer values inside `ref`. */\n void reftable_ref_record_release(struct reftable_ref_record *ref);\n \n /* returns whether two reftable_ref_records are the same. Useful for testing. */\n-int reftable_ref_record_equal(struct reftable_ref_record *a,\n-\t\t\t struct reftable_ref_record *b, int hash_size);\n+int reftable_ref_record_equal(const struct reftable_ref_record *a,\n+\t\t\t const struct reftable_ref_record *b, int hash_size);\n \n /* reftable_log_record holds a reflog entry */\n struct reftable_log_record {\n@@ -104,8 +104,8 @@ int reftable_log_record_is_deletion(const struct reftable_log_record *log);\n void reftable_log_record_release(struct reftable_log_record *log);\n \n /* returns whether two records are equal. Useful for testing. */\n-int reftable_log_record_equal(struct reftable_log_record *a,\n-\t\t\t struct reftable_log_record *b, int hash_size);\n+int reftable_log_record_equal(const struct reftable_log_record *a,\n+\t\t\t const struct reftable_log_record *b, int hash_size);\n \n /* dumps a reftable_log_record on stdout, for debugging/testing. */\n void reftable_log_record_print(struct reftable_log_record *log,\n-- \ngitgitgadget\n\n"},{"id":"446797","messageId":"1e8785bbc2620ce7550f98d5ed6b3761f17c622e.1643051624.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v7.git.git.1643051624.gitgitgadget@gmail.com","subject":"[PATCH v7 15/16] reftable: add print functions to the record types","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-24T19:13:43Z","receivedAt":"2022-01-24T19:39:22Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis isn't used per se, but it is useful for debugging, especially\nWindows CI failures.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/record.c | 88 +++++++++++++++++++++++++++++++++++-------\n reftable/record.h | 4 ++\n reftable/record_test.c | 20 +++++++++-\n 3 files changed, 96 insertions(+), 16 deletions(-)\n\ndiff --git a/reftable/record.c b/reftable/record.c\nindex 238ff882aab..c57bac0e96a 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -269,8 +269,8 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)\n \t}\n }\n \n-void reftable_ref_record_print(const struct reftable_ref_record *ref,\n-\t\t\t uint32_t hash_id)\n+static void reftable_ref_record_print_sz(const struct reftable_ref_record *ref,\n+\t\t\t\t\t int hash_size)\n {\n \tchar hex[GIT_MAX_HEXSZ + 1];\n \tprintf(\"ref{%s(%\" PRIu64 \") \", ref->refname, ref->update_index);\n@@ -279,14 +279,14 @@ void reftable_ref_record_print(const struct reftable_ref_record *ref,\n \t\tprintf(\"=> %s\", ref->value.symref);\n \t\tbreak;\n \tcase REFTABLE_REF_VAL2:\n-\t\thex_format(hex, ref->value.val2.value, hash_size(hash_id));\n+\t\thex_format(hex, ref->value.val2.value, hash_size);\n \t\tprintf(\"val 2 %s\", hex);\n \t\thex_format(hex, ref->value.val2.target_value,\n-\t\t\t hash_size(hash_id));\n+\t\t\t hash_size);\n \t\tprintf(\"(T %s)\", hex);\n \t\tbreak;\n \tcase REFTABLE_REF_VAL1:\n-\t\thex_format(hex, ref->value.val1, hash_size(hash_id));\n+\t\thex_format(hex, ref->value.val1, hash_size);\n \t\tprintf(\"val 1 %s\", hex);\n \t\tbreak;\n \tcase REFTABLE_REF_DELETION:\n@@ -296,6 +296,11 @@ void reftable_ref_record_print(const struct reftable_ref_record *ref,\n \tprintf(\"}\\n\");\n }\n \n+void reftable_ref_record_print(const struct reftable_ref_record *ref,\n+\t\t\t uint32_t hash_id) {\n+\treftable_ref_record_print_sz(ref, hash_size(hash_id));\n+}\n+\n static void reftable_ref_record_release_void(void *rec)\n {\n \treftable_ref_record_release(rec);\n@@ -456,6 +461,13 @@ static int reftable_ref_record_equal_void(const void *a,\n \treturn reftable_ref_record_equal(ra, rb, hash_size);\n }\n \n+static void reftable_ref_record_print_void(const void *rec,\n+\t\t\t\t\t int hash_size)\n+{\n+\treftable_ref_record_print_sz(\n+\t\t(struct reftable_ref_record *) rec, hash_size);\n+}\n+\n static struct reftable_record_vtable reftable_ref_record_vtable = {\n \t.key = &reftable_ref_record_key,\n \t.type = BLOCK_TYPE_REF,\n@@ -466,6 +478,7 @@ static struct reftable_record_vtable reftable_ref_record_vtable = {\n \t.release = &reftable_ref_record_release_void,\n \t.is_deletion = &reftable_ref_record_is_deletion_void,\n \t.equal = &reftable_ref_record_equal_void,\n+\t.print = &reftable_ref_record_print_void,\n };\n \n static void reftable_obj_record_key(const void *r, struct strbuf *dest)\n@@ -484,6 +497,21 @@ static void reftable_obj_record_release(void *rec)\n \tmemset(obj, 0, sizeof(struct reftable_obj_record));\n }\n \n+static void reftable_obj_record_print(const void *rec, int hash_size)\n+{\n+\tconst struct reftable_obj_record *obj = rec;\n+\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 };\n+\tstruct strbuf offset_str = STRBUF_INIT;\n+\tint i;\n+\n+\tfor (i = 0; i < obj->offset_len; i++)\n+\t\tstrbuf_addf(&offset_str, \"%\"PRIu64 \" \", obj->offsets[i]);\n+\thex_format(hex, obj->hash_prefix, obj->hash_prefix_len);\n+\tprintf(\"prefix %s (len %d), offsets [%s]\\n\",\n+\t hex, obj->hash_prefix_len, offset_str.buf);\n+\tstrbuf_release(&offset_str);\n+}\n+\n static void reftable_obj_record_copy_from(void *rec, const void *src_rec,\n \t\t\t\t\t int hash_size)\n {\n@@ -627,31 +655,41 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {\n \t.release = &reftable_obj_record_release,\n \t.is_deletion = ¬_a_deletion,\n \t.equal = &reftable_obj_record_equal_void,\n+\t.print = &reftable_obj_record_print,\n };\n \n-void reftable_log_record_print(struct reftable_log_record *log,\n-\t\t\t uint32_t hash_id)\n+static void reftable_log_record_print_sz(struct reftable_log_record *log,\n+\t\t\t\t\t int hash_size)\n {\n \tchar hex[GIT_MAX_HEXSZ + 1] = { 0 };\n \n \tswitch (log->value_type) {\n \tcase REFTABLE_LOG_DELETION:\n-\t\tprintf(\"log{%s(%\" PRIu64 \") delete\", log->refname,\n+\t\tprintf(\"log{%s(%\"PRIu64 \") delete\\n\", log->refname,\n \t\t log->update_index);\n \t\tbreak;\n \tcase REFTABLE_LOG_UPDATE:\n-\t\tprintf(\"log{%s(%\" PRIu64 \") %s <%s> %\" PRIu64 \" %04d\\n\",\n-\t\t log->refname, log->update_index, log->value.update.name,\n-\t\t log->value.update.email, log->value.update.time,\n+\t\tprintf(\"log{%s(%\"PRIu64 \") %s <%s> %\" PRIu64 \" %04d\\n\",\n+\t\t log->refname, log->update_index,\n+\t\t log->value.update.name ? log->value.update.name : \"\",\n+\t\t log->value.update.email ? log->value.update.email : \"\",\n+\t\t log->value.update.time,\n \t\t log->value.update.tz_offset);\n-\t\thex_format(hex, log->value.update.old_hash, hash_size(hash_id));\n+\t\thex_format(hex, log->value.update.old_hash, hash_size);\n \t\tprintf(\"%s => \", hex);\n-\t\thex_format(hex, log->value.update.new_hash, hash_size(hash_id));\n-\t\tprintf(\"%s\\n\\n%s\\n}\\n\", hex, log->value.update.message);\n+\t\thex_format(hex, log->value.update.new_hash, hash_size);\n+\t\tprintf(\"%s\\n\\n%s\\n}\\n\", hex,\n+\t\t log->value.update.message ? log->value.update.message : \"\");\n \t\tbreak;\n \t}\n }\n \n+void reftable_log_record_print(struct reftable_log_record *log,\n+\t\t\t\t uint32_t hash_id)\n+{\n+\treftable_log_record_print_sz(log, hash_size(hash_id));\n+}\n+\n static void reftable_log_record_key(const void *r, struct strbuf *dest)\n {\n \tconst struct reftable_log_record *rec =\n@@ -969,6 +1007,11 @@ static int reftable_log_record_is_deletion_void(const void *p)\n \t\t(const struct reftable_log_record *)p);\n }\n \n+static void reftable_log_record_print_void(const void *rec, int hash_size)\n+{\n+\treftable_log_record_print_sz((struct reftable_log_record*)rec, hash_size);\n+}\n+\n static struct reftable_record_vtable reftable_log_record_vtable = {\n \t.key = &reftable_log_record_key,\n \t.type = BLOCK_TYPE_LOG,\n@@ -978,7 +1021,8 @@ static struct reftable_record_vtable reftable_log_record_vtable = {\n \t.decode = &reftable_log_record_decode,\n \t.release = &reftable_log_record_release_void,\n \t.is_deletion = &reftable_log_record_is_deletion_void,\n-\t.equal = &reftable_log_record_equal_void\n+\t.equal = &reftable_log_record_equal_void,\n+\t.print = &reftable_log_record_print_void,\n };\n \n static void reftable_index_record_key(const void *r, struct strbuf *dest)\n@@ -1054,6 +1098,13 @@ static int reftable_index_record_equal(const void *a, const void *b, int hash_si\n \treturn ia->offset == ib->offset && !strbuf_cmp(&ia->last_key, &ib->last_key);\n }\n \n+static void reftable_index_record_print(const void *rec, int hash_size)\n+{\n+\tconst struct reftable_index_record *idx = rec;\n+\t/* TODO: escape null chars? */\n+\tprintf(\"\\\"%s\\\" %\" PRIu64 \"\\n\", idx->last_key.buf, idx->offset);\n+}\n+\n static struct reftable_record_vtable reftable_index_record_vtable = {\n \t.key = &reftable_index_record_key,\n \t.type = BLOCK_TYPE_INDEX,\n@@ -1064,6 +1115,7 @@ static struct reftable_record_vtable reftable_index_record_vtable = {\n \t.release = &reftable_index_record_release,\n \t.is_deletion = ¬_a_deletion,\n \t.equal = &reftable_index_record_equal,\n+\t.print = &reftable_index_record_print,\n };\n \n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)\n@@ -1251,3 +1303,9 @@ struct reftable_record reftable_new_record(uint8_t type)\n \t}\n \treturn clean;\n }\n+\n+void reftable_record_print(struct reftable_record *rec, int hash_size)\n+{\n+\tprintf(\"'%c': \", rec->type);\n+\treftable_record_vtable(rec)->print(reftable_record_data(rec), hash_size);\n+}\ndiff --git a/reftable/record.h b/reftable/record.h\nindex 010a322e901..fd80cd451d5 100644\n--- a/reftable/record.h\n+++ b/reftable/record.h\n@@ -61,6 +61,9 @@ struct reftable_record_vtable {\n \n \t/* Are two records equal? This assumes they have the same type. Returns 0 for non-equal. */\n \tint (*equal)(const void *a, const void *b, int hash_size);\n+\n+\t/* Print on stdout, for debugging. */\n+\tvoid (*print)(const void *rec, int hash_size);\n };\n \n /* returns true for recognized block types. Block start with the block type. */\n@@ -112,6 +115,7 @@ struct reftable_record {\n \n /* see struct record_vtable */\n int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size);\n+void reftable_record_print(struct reftable_record *rec, int hash_size);\n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest);\n uint8_t reftable_record_type(struct reftable_record *rec);\n void reftable_record_copy_from(struct reftable_record *rec,\ndiff --git a/reftable/record_test.c b/reftable/record_test.c\nindex 4a025c48104..847abfa2b04 100644\n--- a/reftable/record_test.c\n+++ b/reftable/record_test.c\n@@ -23,6 +23,10 @@ static void test_copy(struct reftable_record *rec)\n \t/* do it twice to catch memory leaks */\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \tEXPECT(reftable_record_equal(rec, ©, GIT_SHA1_RAWSZ));\n+\n+\tputs(\"testing print coverage:\\n\");\n+\treftable_record_print(©, GIT_SHA1_RAWSZ);\n+\n \treftable_record_release(©);\n }\n \n@@ -175,7 +179,7 @@ static void test_reftable_log_record_equal(void)\n static void test_reftable_log_record_roundtrip(void)\n {\n \tint i;\n-\tstruct reftable_log_record in[2] = {\n+\tstruct reftable_log_record in[] = {\n \t\t{\n \t\t\t.refname = xstrdup(\"refs/heads/master\"),\n \t\t\t.update_index = 42,\n@@ -196,10 +200,24 @@ static void test_reftable_log_record_roundtrip(void)\n \t\t\t.refname = xstrdup(\"refs/heads/master\"),\n \t\t\t.update_index = 22,\n \t\t\t.value_type = REFTABLE_LOG_DELETION,\n+\t\t},\n+\t\t{\n+\t\t\t/* make sure we handle optional fields correctly. */\n+\t\t\t.refname = xstrdup(\"branch\"),\n+\t\t\t.update_index = 33,\n+\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n+\t\t\t.value = {\n+\t\t\t\t.update = {\n+\t\t\t\t\t.old_hash = reftable_malloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t.new_hash = reftable_malloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t},\n+\t\t\t},\n \t\t}\n \t};\n \tset_test_hash(in[0].value.update.new_hash, 1);\n \tset_test_hash(in[0].value.update.old_hash, 2);\n+\tset_test_hash(in[2].value.update.new_hash, 3);\n+\tset_test_hash(in[2].value.update.old_hash, 4);\n \tfor (i = 0; i < ARRAY_SIZE(in); i++) {\n \t\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG };\n \t\tstruct strbuf key = STRBUF_INIT;\n-- \ngitgitgadget\n\n"},{"id":"446799","messageId":"047bbb75fcbb67c41d307bf5fa44587a76cc0777.1643051624.git.gitgitgadget@gmail.com","threadId":"57047","inReplyTo":"pull.1152.v7.git.git.1643051624.gitgitgadget@gmail.com","subject":"[PATCH v7 14/16] reftable: make reftable_record a tagged union","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-24T19:13:42Z","receivedAt":"2022-01-24T19:40:44Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis reduces the amount of glue code, because we don't need a void\npointer or vtable within the structure.\n\nThe only snag is that reftable_index_record contain a strbuf, so it\ncannot be zero-initialized. To address this, use reftable_new_record()\nto return fresh instance, given a record type. Since\nreftable_new_record() doesn't cause heap allocation anymore, it should\nbe balanced with reftable_record_release() rather than\nreftable_record_destroy().\n\nThanks to Peff for the suggestion.\n\nHelped-by: Jeff King \nSigned-off-by: Han-Wen Nienhuys \n---\n reftable/block.c | 2 +-\n reftable/block_test.c | 22 ++---\n reftable/generic.c | 39 +++++---\n reftable/iter.c | 4 +-\n reftable/merged.c | 31 +++---\n reftable/pq.c | 2 +-\n reftable/pq_test.c | 29 +++---\n reftable/reader.c | 84 +++++++++--------\n reftable/record.c | 209 +++++++++++++++++++----------------------\n reftable/record.h | 42 ++++-----\n reftable/record_test.c | 154 +++++++++++++++---------------\n reftable/writer.c | 38 ++++----\n 12 files changed, 328 insertions(+), 328 deletions(-)\n\ndiff --git a/reftable/block.c b/reftable/block.c\nindex 5832f85501d..80ad10bb794 100644\n--- a/reftable/block.c\n+++ b/reftable/block.c\n@@ -422,7 +422,7 @@ int block_reader_seek(struct block_reader *br, struct block_iter *it,\n done:\n \tstrbuf_release(&key);\n \tstrbuf_release(&next.last_key);\n-\treftable_record_destroy(&rec);\n+\treftable_record_release(&rec);\n \n \treturn err;\n }\ndiff --git a/reftable/block_test.c b/reftable/block_test.c\nindex 4b3ea262dcb..fa2ee092ec0 100644\n--- a/reftable/block_test.c\n+++ b/reftable/block_test.c\n@@ -26,8 +26,9 @@ static void test_block_read_write(void)\n \tstruct block_writer bw = {\n \t\t.last_key = STRBUF_INIT,\n \t};\n-\tstruct reftable_ref_record ref = { NULL };\n-\tstruct reftable_record rec = { NULL };\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t};\n \tint i = 0;\n \tint n;\n \tstruct block_reader br = { 0 };\n@@ -40,7 +41,6 @@ static void test_block_read_write(void)\n \tblock.source = malloc_block_source();\n \tblock_writer_init(&bw, BLOCK_TYPE_REF, block.data, block_size,\n \t\t\t header_off, hash_size(GIT_SHA1_FORMAT_ID));\n-\treftable_record_from_ref(&rec, &ref);\n \n \tfor (i = 0; i < N; i++) {\n \t\tchar name[100];\n@@ -48,14 +48,14 @@ static void test_block_read_write(void)\n \t\tsnprintf(name, sizeof(name), \"branch%02d\", i);\n \t\tmemset(hash, i, sizeof(hash));\n \n-\t\tref.refname = name;\n-\t\tref.value_type = REFTABLE_REF_VAL1;\n-\t\tref.value.val1 = hash;\n+\t\trec.u.ref.refname = name;\n+\t\trec.u.ref.value_type = REFTABLE_REF_VAL1;\n+\t\trec.u.ref.value.val1 = hash;\n \n \t\tnames[i] = xstrdup(name);\n \t\tn = block_writer_add(&bw, &rec);\n-\t\tref.refname = NULL;\n-\t\tref.value_type = REFTABLE_REF_DELETION;\n+\t\trec.u.ref.refname = NULL;\n+\t\trec.u.ref.value_type = REFTABLE_REF_DELETION;\n \t\tEXPECT(n == 0);\n \t}\n \n@@ -74,7 +74,7 @@ static void test_block_read_write(void)\n \t\tif (r > 0) {\n \t\t\tbreak;\n \t\t}\n-\t\tEXPECT_STREQ(names[j], ref.refname);\n+\t\tEXPECT_STREQ(names[j], rec.u.ref.refname);\n \t\tj++;\n \t}\n \n@@ -92,7 +92,7 @@ static void test_block_read_write(void)\n \t\tn = block_iter_next(&it, &rec);\n \t\tEXPECT(n == 0);\n \n-\t\tEXPECT_STREQ(names[i], ref.refname);\n+\t\tEXPECT_STREQ(names[i], rec.u.ref.refname);\n \n \t\twant.len--;\n \t\tn = block_reader_seek(&br, &it, &want);\n@@ -100,7 +100,7 @@ static void test_block_read_write(void)\n \n \t\tn = block_iter_next(&it, &rec);\n \t\tEXPECT(n == 0);\n-\t\tEXPECT_STREQ(names[10 * (i / 10)], ref.refname);\n+\t\tEXPECT_STREQ(names[10 * (i / 10)], rec.u.ref.refname);\n \n \t\tblock_iter_close(&it);\n \t}\ndiff --git a/reftable/generic.c b/reftable/generic.c\nindex 7a8a738d860..b176c1cde0a 100644\n--- a/reftable/generic.c\n+++ b/reftable/generic.c\n@@ -7,6 +7,7 @@ https://developers.google.com/open-source/licenses/bsd\n */\n \n #include \"basics.h\"\n+#include \"constants.h\"\n #include \"record.h\"\n #include \"generic.h\"\n #include \"reftable-iterator.h\"\n@@ -15,23 +16,23 @@ https://developers.google.com/open-source/licenses/bsd\n int reftable_table_seek_ref(struct reftable_table *tab,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = { .refname = (char *)name, }\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n }\n \n int reftable_table_seek_log(struct reftable_table *tab,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = ~((uint64_t)0),\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.u.log = {\n+\t\t\t.refname = (char *)name,\n+\t\t\t.update_index = ~((uint64_t)0),\n+\t\t}\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n \treturn tab->ops->seek_record(tab->table_arg, it, &rec);\n }\n \n@@ -129,17 +130,25 @@ void reftable_iterator_destroy(struct reftable_iterator *it)\n int reftable_iterator_next_ref(struct reftable_iterator *it,\n \t\t\t struct reftable_ref_record *ref)\n {\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, ref);\n-\treturn iterator_next(it, &rec);\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = *ref,\n+\t};\n+\tint err = iterator_next(it, &rec);\n+\t*ref = rec.u.ref;\n+\treturn err;\n }\n \n int reftable_iterator_next_log(struct reftable_iterator *it,\n \t\t\t struct reftable_log_record *log)\n {\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, log);\n-\treturn iterator_next(it, &rec);\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.u.log = *log,\n+\t};\n+\tint err = iterator_next(it, &rec);\n+\t*log = rec.u.log;\n+\treturn err;\n }\n \n int iterator_next(struct reftable_iterator *it, struct reftable_record *rec)\ndiff --git a/reftable/iter.c b/reftable/iter.c\nindex 93d04f735b8..a8d174c0406 100644\n--- a/reftable/iter.c\n+++ b/reftable/iter.c\n@@ -32,7 +32,7 @@ static int filtering_ref_iterator_next(void *iter_arg,\n \t\t\t\t struct reftable_record *rec)\n {\n \tstruct filtering_ref_iterator *fri = iter_arg;\n-\tstruct reftable_ref_record *ref = rec->data;\n+\tstruct reftable_ref_record *ref = &rec->u.ref;\n \tint err = 0;\n \twhile (1) {\n \t\terr = reftable_iterator_next_ref(&fri->it, ref);\n@@ -127,7 +127,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)\n static int indexed_table_ref_iter_next(void *p, struct reftable_record *rec)\n {\n \tstruct indexed_table_ref_iter *it = p;\n-\tstruct reftable_ref_record *ref = rec->data;\n+\tstruct reftable_ref_record *ref = &rec->u.ref;\n \n \twhile (1) {\n \t\tint err = block_iter_next(&it->cur, rec);\ndiff --git a/reftable/merged.c b/reftable/merged.c\nindex e5b53da6db3..82511762d2a 100644\n--- a/reftable/merged.c\n+++ b/reftable/merged.c\n@@ -30,7 +30,7 @@ static int merged_iter_init(struct merged_iter *mi)\n \n \t\tif (err > 0) {\n \t\t\treftable_iterator_destroy(&mi->stack[i]);\n-\t\t\treftable_record_destroy(&rec);\n+\t\t\treftable_record_release(&rec);\n \t\t} else {\n \t\t\tstruct pq_entry e = {\n \t\t\t\t.rec = rec,\n@@ -57,18 +57,17 @@ static void merged_iter_close(void *p)\n static int merged_iter_advance_nonnull_subiter(struct merged_iter *mi,\n \t\t\t\t\t size_t idx)\n {\n-\tstruct reftable_record rec = reftable_new_record(mi->typ);\n \tstruct pq_entry e = {\n-\t\t.rec = rec,\n+\t\t.rec = reftable_new_record(mi->typ),\n \t\t.index = idx,\n \t};\n-\tint err = iterator_next(&mi->stack[idx], &rec);\n+\tint err = iterator_next(&mi->stack[idx], &e.rec);\n \tif (err < 0)\n \t\treturn err;\n \n \tif (err > 0) {\n \t\treftable_iterator_destroy(&mi->stack[idx]);\n-\t\treftable_record_destroy(&rec);\n+\t\treftable_record_release(&e.rec);\n \t\treturn 0;\n \t}\n \n@@ -126,11 +125,11 @@ static int merged_iter_next_entry(struct merged_iter *mi,\n \t\tif (err < 0) {\n \t\t\treturn err;\n \t\t}\n-\t\treftable_record_destroy(&top.rec);\n+\t\treftable_record_release(&top.rec);\n \t}\n \n \treftable_record_copy_from(rec, &entry.rec, hash_size(mi->hash_id));\n-\treftable_record_destroy(&entry.rec);\n+\treftable_record_release(&entry.rec);\n \tstrbuf_release(&entry_key);\n \treturn 0;\n }\n@@ -290,11 +289,10 @@ int reftable_merged_table_seek_ref(struct reftable_merged_table *mt,\n \t\t\t\t struct reftable_iterator *it,\n \t\t\t\t const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = { .refname = (char *)name, },\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n \treturn merged_table_seek_record(mt, it, &rec);\n }\n \n@@ -302,12 +300,13 @@ int reftable_merged_table_seek_log_at(struct reftable_merged_table *mt,\n \t\t\t\t struct reftable_iterator *it,\n \t\t\t\t const char *name, uint64_t update_index)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = update_index,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.u.log = {\n+\t\t\t.refname = (char *)name,\n+\t\t\t.update_index = update_index,\n+\t\t}\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n \treturn merged_table_seek_record(mt, it, &rec);\n }\n \ndiff --git a/reftable/pq.c b/reftable/pq.c\nindex efc474017a2..5ffc8380c4a 100644\n--- a/reftable/pq.c\n+++ b/reftable/pq.c\n@@ -98,7 +98,7 @@ void merged_iter_pqueue_release(struct merged_iter_pqueue *pq)\n {\n \tint i = 0;\n \tfor (i = 0; i < pq->len; i++) {\n-\t\treftable_record_destroy(&pq->heap[i].rec);\n+\t\treftable_record_release(&pq->heap[i].rec);\n \t}\n \tFREE_AND_NULL(pq->heap);\n \tpq->len = pq->cap = 0;\ndiff --git a/reftable/pq_test.c b/reftable/pq_test.c\nindex c9bb05e37b7..5d58a4c05b6 100644\n--- a/reftable/pq_test.c\n+++ b/reftable/pq_test.c\n@@ -31,7 +31,7 @@ static void test_pq(void)\n \tint N = ARRAY_SIZE(names) - 1;\n \n \tstruct merged_iter_pqueue pq = { NULL };\n-\tconst char *last = NULL;\n+\tchar *last = NULL;\n \n \tint i = 0;\n \tfor (i = 0; i < N; i++) {\n@@ -42,12 +42,12 @@ static void test_pq(void)\n \n \ti = 1;\n \tdo {\n-\t\tstruct reftable_record rec =\n-\t\t\treftable_new_record(BLOCK_TYPE_REF);\n-\t\tstruct pq_entry e = { 0 };\n-\n-\t\treftable_record_as_ref(&rec)->refname = names[i];\n-\t\te.rec = rec;\n+\t\tstruct pq_entry e = {\n+\t\t\t.rec = {\n+\t\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t\t\t.u.ref = { .refname = names[i], }\n+\t\t\t}\n+\t\t};\n \t\tmerged_iter_pqueue_add(&pq, e);\n \t\tmerged_iter_pqueue_check(pq);\n \t\ti = (i * 7) % N;\n@@ -55,19 +55,18 @@ static void test_pq(void)\n \n \twhile (!merged_iter_pqueue_is_empty(pq)) {\n \t\tstruct pq_entry e = merged_iter_pqueue_remove(&pq);\n-\t\tstruct reftable_ref_record *ref =\n-\t\t\treftable_record_as_ref(&e.rec);\n-\n+\t\tstruct reftable_record *rec = &e.rec;\n \t\tmerged_iter_pqueue_check(pq);\n \n+\t\tEXPECT(reftable_record_type(rec) == BLOCK_TYPE_REF);\n \t\tif (last) {\n-\t\t\tEXPECT(strcmp(last, ref->refname) < 0);\n+\t\t\tEXPECT(strcmp(last, rec->u.ref.refname) < 0);\n \t\t}\n-\t\tlast = ref->refname;\n-\t\tref->refname = NULL;\n-\t\treftable_free(ref);\n+\t\t/* this is names[i], so don't dealloc. */\n+\t\tlast = rec->u.ref.refname;\n+\t\trec->u.ref.refname = NULL;\n+\t\treftable_record_release(rec);\n \t}\n-\n \tfor (i = 0; i < N; i++) {\n \t\treftable_free(names[i]);\n \t}\ndiff --git a/reftable/reader.c b/reftable/reader.c\nindex 272378ed1d5..565cda8d5a2 100644\n--- a/reftable/reader.c\n+++ b/reftable/reader.c\n@@ -239,8 +239,7 @@ static int table_iter_next_in_block(struct table_iter *ti,\n {\n \tint res = block_iter_next(&ti->bi, rec);\n \tif (res == 0 && reftable_record_type(rec) == BLOCK_TYPE_REF) {\n-\t\t((struct reftable_ref_record *)rec->data)->update_index +=\n-\t\t\tti->r->min_update_index;\n+\t\trec->u.ref.update_index += ti->r->min_update_index;\n \t}\n \n \treturn res;\n@@ -480,7 +479,7 @@ static int reader_seek_linear(struct reftable_reader *r, struct table_iter *ti,\n \n done:\n \tblock_iter_close(&next.bi);\n-\treftable_record_destroy(&rec);\n+\treftable_record_release(&rec);\n \tstrbuf_release(&want_key);\n \tstrbuf_release(&got_key);\n \treturn err;\n@@ -490,34 +489,36 @@ static int reader_seek_indexed(struct reftable_reader *r,\n \t\t\t struct reftable_iterator *it,\n \t\t\t struct reftable_record *rec)\n {\n-\tstruct reftable_index_record want_index = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record want_index_rec = { NULL };\n-\tstruct reftable_index_record index_result = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record index_result_rec = { NULL };\n+\tstruct reftable_record want_index = {\n+\t\t.type = BLOCK_TYPE_INDEX, .u.idx = { .last_key = STRBUF_INIT }\n+\t};\n+\tstruct reftable_record index_result = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = { .last_key = STRBUF_INIT },\n+\t};\n \tstruct table_iter index_iter = TABLE_ITER_INIT;\n \tstruct table_iter next = TABLE_ITER_INIT;\n \tint err = 0;\n \n-\treftable_record_key(rec, &want_index.last_key);\n-\treftable_record_from_index(&want_index_rec, &want_index);\n-\treftable_record_from_index(&index_result_rec, &index_result);\n+\treftable_record_key(rec, &want_index.u.idx.last_key);\n \n \terr = reader_start(r, &index_iter, reftable_record_type(rec), 1);\n \tif (err < 0)\n \t\tgoto done;\n \n-\terr = reader_seek_linear(r, &index_iter, &want_index_rec);\n+\terr = reader_seek_linear(r, &index_iter, &want_index);\n \twhile (1) {\n-\t\terr = table_iter_next(&index_iter, &index_result_rec);\n+\t\terr = table_iter_next(&index_iter, &index_result);\n \t\ttable_iter_block_done(&index_iter);\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = reader_table_iter_at(r, &next, index_result.offset, 0);\n+\t\terr = reader_table_iter_at(r, &next, index_result.u.idx.offset,\n+\t\t\t\t\t 0);\n \t\tif (err != 0)\n \t\t\tgoto done;\n \n-\t\terr = block_iter_seek(&next.bi, &want_index.last_key);\n+\t\terr = block_iter_seek(&next.bi, &want_index.u.idx.last_key);\n \t\tif (err < 0)\n \t\t\tgoto done;\n \n@@ -545,8 +546,8 @@ static int reader_seek_indexed(struct reftable_reader *r,\n done:\n \tblock_iter_close(&next.bi);\n \ttable_iter_close(&index_iter);\n-\treftable_record_release(&want_index_rec);\n-\treftable_record_release(&index_result_rec);\n+\treftable_record_release(&want_index);\n+\treftable_record_release(&index_result);\n \treturn err;\n }\n \n@@ -595,11 +596,10 @@ static int reader_seek(struct reftable_reader *r, struct reftable_iterator *it,\n int reftable_reader_seek_ref(struct reftable_reader *r,\n \t\t\t struct reftable_iterator *it, const char *name)\n {\n-\tstruct reftable_ref_record ref = {\n-\t\t.refname = (char *)name,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = { .refname = (char *)name, },\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_ref(&rec, &ref);\n \treturn reader_seek(r, it, &rec);\n }\n \n@@ -607,12 +607,13 @@ int reftable_reader_seek_log_at(struct reftable_reader *r,\n \t\t\t\tstruct reftable_iterator *it, const char *name,\n \t\t\t\tuint64_t update_index)\n {\n-\tstruct reftable_log_record log = {\n-\t\t.refname = (char *)name,\n-\t\t.update_index = update_index,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.u.log = {\n+\t\t\t.refname = (char *)name,\n+\t\t\t.update_index = update_index,\n+\t\t}\n \t};\n-\tstruct reftable_record rec = { NULL };\n-\treftable_record_from_log(&rec, &log);\n \treturn reader_seek(r, it, &rec);\n }\n \n@@ -656,31 +657,33 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \t\t\t\t\t struct reftable_iterator *it,\n \t\t\t\t\t uint8_t *oid)\n {\n-\tstruct reftable_obj_record want = {\n-\t\t.hash_prefix = oid,\n-\t\t.hash_prefix_len = r->object_id_len,\n+\tstruct reftable_record want = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.u.obj = {\n+\t\t\t.hash_prefix = oid,\n+\t\t\t.hash_prefix_len = r->object_id_len,\n+\t\t},\n \t};\n-\tstruct reftable_record want_rec = { NULL };\n \tstruct reftable_iterator oit = { NULL };\n-\tstruct reftable_obj_record got = { NULL };\n-\tstruct reftable_record got_rec = { NULL };\n+\tstruct reftable_record got = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.u.obj = { 0 },\n+\t};\n \tint err = 0;\n \tstruct indexed_table_ref_iter *itr = NULL;\n \n \t/* Look through the reverse index. */\n-\treftable_record_from_obj(&want_rec, &want);\n-\terr = reader_seek(r, &oit, &want_rec);\n+\terr = reader_seek(r, &oit, &want);\n \tif (err != 0)\n \t\tgoto done;\n \n \t/* read out the reftable_obj_record */\n-\treftable_record_from_obj(&got_rec, &got);\n-\terr = iterator_next(&oit, &got_rec);\n+\terr = iterator_next(&oit, &got);\n \tif (err < 0)\n \t\tgoto done;\n \n-\tif (err > 0 ||\n-\t memcmp(want.hash_prefix, got.hash_prefix, r->object_id_len)) {\n+\tif (err > 0 || memcmp(want.u.obj.hash_prefix, got.u.obj.hash_prefix,\n+\t\t\t r->object_id_len)) {\n \t\t/* didn't find it; return empty iterator */\n \t\titerator_set_empty(it);\n \t\terr = 0;\n@@ -688,15 +691,16 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r,\n \t}\n \n \terr = new_indexed_table_ref_iter(&itr, r, oid, hash_size(r->hash_id),\n-\t\t\t\t\t got.offsets, got.offset_len);\n+\t\t\t\t\t got.u.obj.offsets,\n+\t\t\t\t\t got.u.obj.offset_len);\n \tif (err < 0)\n \t\tgoto done;\n-\tgot.offsets = NULL;\n+\tgot.u.obj.offsets = NULL;\n \titerator_from_indexed_table_ref_iter(it, itr);\n \n done:\n \treftable_iterator_destroy(&oit);\n-\treftable_record_release(&got_rec);\n+\treftable_record_release(&got);\n \treturn err;\n }\n \ndiff --git a/reftable/record.c b/reftable/record.c\nindex b543585ad39..238ff882aab 100644\n--- a/reftable/record.c\n+++ b/reftable/record.c\n@@ -15,6 +15,24 @@ https://developers.google.com/open-source/licenses/bsd\n #include \"reftable-error.h\"\n #include \"basics.h\"\n \n+static struct reftable_record_vtable *\n+reftable_record_vtable(struct reftable_record *rec);\n+\n+static void *reftable_record_data(struct reftable_record *rec)\n+{\n+\tswitch (rec->type) {\n+\tcase BLOCK_TYPE_REF:\n+\t\treturn &rec->u.ref;\n+\tcase BLOCK_TYPE_LOG:\n+\t\treturn &rec->u.log;\n+\tcase BLOCK_TYPE_INDEX:\n+\t\treturn &rec->u.idx;\n+\tcase BLOCK_TYPE_OBJ:\n+\t\treturn &rec->u.obj;\n+\t}\n+\tabort();\n+}\n+\n int get_var_int(uint64_t *dest, struct string_view *in)\n {\n \tint ptr = 0;\n@@ -474,12 +492,13 @@ static void reftable_obj_record_copy_from(void *rec, const void *src_rec,\n \t\t(const struct reftable_obj_record *)src_rec;\n \n \treftable_obj_record_release(obj);\n-\t*obj = *src;\n-\tobj->hash_prefix = reftable_malloc(obj->hash_prefix_len);\n+\tobj->hash_prefix = reftable_malloc(src->hash_prefix_len);\n+\tobj->hash_prefix_len = src->hash_prefix_len;\n \tmemcpy(obj->hash_prefix, src->hash_prefix, obj->hash_prefix_len);\n \n-\tobj->offsets = reftable_malloc(obj->offset_len * sizeof(uint64_t));\n-\tCOPY_ARRAY(obj->offsets, src->offsets, obj->offset_len);\n+\tobj->offsets = reftable_malloc(src->offset_len * sizeof(uint64_t));\n+\tobj->offset_len = src->offset_len;\n+\tCOPY_ARRAY(obj->offsets, src->offsets, src->offset_len);\n }\n \n static uint8_t reftable_obj_record_val_type(const void *rec)\n@@ -962,58 +981,6 @@ static struct reftable_record_vtable reftable_log_record_vtable = {\n \t.equal = &reftable_log_record_equal_void\n };\n \n-struct reftable_record reftable_new_record(uint8_t typ)\n-{\n-\tstruct reftable_record rec = { NULL };\n-\tswitch (typ) {\n-\tcase BLOCK_TYPE_REF: {\n-\t\tstruct reftable_ref_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_ref_record));\n-\t\treftable_record_from_ref(&rec, r);\n-\t\treturn rec;\n-\t}\n-\n-\tcase BLOCK_TYPE_OBJ: {\n-\t\tstruct reftable_obj_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_obj_record));\n-\t\treftable_record_from_obj(&rec, r);\n-\t\treturn rec;\n-\t}\n-\tcase BLOCK_TYPE_LOG: {\n-\t\tstruct reftable_log_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_log_record));\n-\t\treftable_record_from_log(&rec, r);\n-\t\treturn rec;\n-\t}\n-\tcase BLOCK_TYPE_INDEX: {\n-\t\tstruct reftable_index_record empty = { .last_key =\n-\t\t\t\t\t\t\t STRBUF_INIT };\n-\t\tstruct reftable_index_record *r =\n-\t\t\treftable_calloc(sizeof(struct reftable_index_record));\n-\t\t*r = empty;\n-\t\treftable_record_from_index(&rec, r);\n-\t\treturn rec;\n-\t}\n-\t}\n-\tabort();\n-\treturn rec;\n-}\n-\n-/* clear out the record, yielding the reftable_record data that was\n- * encapsulated. */\n-static void *reftable_record_yield(struct reftable_record *rec)\n-{\n-\tvoid *p = rec->data;\n-\trec->data = NULL;\n-\treturn p;\n-}\n-\n-void reftable_record_destroy(struct reftable_record *rec)\n-{\n-\treftable_record_release(rec);\n-\treftable_free(reftable_record_yield(rec));\n-}\n-\n static void reftable_index_record_key(const void *r, struct strbuf *dest)\n {\n \tconst struct reftable_index_record *rec = r;\n@@ -1036,6 +1003,7 @@ static void reftable_index_record_release(void *rec)\n {\n \tstruct reftable_index_record *idx = rec;\n \tstrbuf_release(&idx->last_key);\n+\tidx->offset = 0;\n }\n \n static uint8_t reftable_index_record_val_type(const void *rec)\n@@ -1100,98 +1068,60 @@ static struct reftable_record_vtable reftable_index_record_vtable = {\n \n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)\n {\n-\trec->ops->key(rec->data, dest);\n+\treftable_record_vtable(rec)->key(reftable_record_data(rec), dest);\n }\n \n uint8_t reftable_record_type(struct reftable_record *rec)\n {\n-\treturn rec->ops->type;\n+\treturn rec->type;\n }\n \n int reftable_record_encode(struct reftable_record *rec, struct string_view dest,\n \t\t\t int hash_size)\n {\n-\treturn rec->ops->encode(rec->data, dest, hash_size);\n+\treturn reftable_record_vtable(rec)->encode(reftable_record_data(rec),\n+\t\t\t\t\t\t dest, hash_size);\n }\n \n void reftable_record_copy_from(struct reftable_record *rec,\n \t\t\t struct reftable_record *src, int hash_size)\n {\n-\tassert(src->ops->type == rec->ops->type);\n+\tassert(src->type == rec->type);\n \n-\trec->ops->copy_from(rec->data, src->data, hash_size);\n+\treftable_record_vtable(rec)->copy_from(reftable_record_data(rec),\n+\t\t\t\t\t reftable_record_data(src),\n+\t\t\t\t\t hash_size);\n }\n \n uint8_t reftable_record_val_type(struct reftable_record *rec)\n {\n-\treturn rec->ops->val_type(rec->data);\n+\treturn reftable_record_vtable(rec)->val_type(reftable_record_data(rec));\n }\n \n int reftable_record_decode(struct reftable_record *rec, struct strbuf key,\n \t\t\t uint8_t extra, struct string_view src, int hash_size)\n {\n-\treturn rec->ops->decode(rec->data, key, extra, src, hash_size);\n+\treturn reftable_record_vtable(rec)->decode(reftable_record_data(rec),\n+\t\t\t\t\t\t key, extra, src, hash_size);\n }\n \n void reftable_record_release(struct reftable_record *rec)\n {\n-\trec->ops->release(rec->data);\n+\treftable_record_vtable(rec)->release(reftable_record_data(rec));\n }\n \n int reftable_record_is_deletion(struct reftable_record *rec)\n {\n-\treturn rec->ops->is_deletion(rec->data);\n+\treturn reftable_record_vtable(rec)->is_deletion(\n+\t\treftable_record_data(rec));\n }\n \n int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size)\n {\n-\tif (a->ops != b->ops)\n+\tif (a->type != b->type)\n \t\treturn 0;\n-\treturn a->ops->equal(a->data, b->data, hash_size);\n-}\n-\n-void reftable_record_from_ref(struct reftable_record *rec,\n-\t\t\t struct reftable_ref_record *ref_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = ref_rec;\n-\trec->ops = &reftable_ref_record_vtable;\n-}\n-\n-void reftable_record_from_obj(struct reftable_record *rec,\n-\t\t\t struct reftable_obj_record *obj_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = obj_rec;\n-\trec->ops = &reftable_obj_record_vtable;\n-}\n-\n-void reftable_record_from_index(struct reftable_record *rec,\n-\t\t\t\tstruct reftable_index_record *index_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = index_rec;\n-\trec->ops = &reftable_index_record_vtable;\n-}\n-\n-void reftable_record_from_log(struct reftable_record *rec,\n-\t\t\t struct reftable_log_record *log_rec)\n-{\n-\tassert(!rec->ops);\n-\trec->data = log_rec;\n-\trec->ops = &reftable_log_record_vtable;\n-}\n-\n-struct reftable_ref_record *reftable_record_as_ref(struct reftable_record *rec)\n-{\n-\tassert(reftable_record_type(rec) == BLOCK_TYPE_REF);\n-\treturn rec->data;\n-}\n-\n-struct reftable_log_record *reftable_record_as_log(struct reftable_record *rec)\n-{\n-\tassert(reftable_record_type(rec) == BLOCK_TYPE_LOG);\n-\treturn rec->data;\n+\treturn reftable_record_vtable(a)->equal(\n+\t\treftable_record_data(a), reftable_record_data(b), hash_size);\n }\n \n static int hash_equal(uint8_t *a, uint8_t *b, int hash_size)\n@@ -1264,3 +1194,60 @@ void string_view_consume(struct string_view *s, int n)\n \ts->buf += n;\n \ts->len -= n;\n }\n+\n+static struct reftable_record_vtable *\n+reftable_record_vtable(struct reftable_record *rec)\n+{\n+\tswitch (rec->type) {\n+\tcase BLOCK_TYPE_REF:\n+\t\treturn &reftable_ref_record_vtable;\n+\tcase BLOCK_TYPE_LOG:\n+\t\treturn &reftable_log_record_vtable;\n+\tcase BLOCK_TYPE_INDEX:\n+\t\treturn &reftable_index_record_vtable;\n+\tcase BLOCK_TYPE_OBJ:\n+\t\treturn &reftable_obj_record_vtable;\n+\t}\n+\tabort();\n+}\n+\n+struct reftable_record reftable_new_record(uint8_t type)\n+{\n+\tstruct reftable_record clean = {\n+\t\t.type = type,\n+\t};\n+\n+\t/* the following is involved, but the naive solution (just return\n+\t * `clean` as is, except for BLOCK_TYPE_INDEX), returns a garbage\n+\t * clean.u.obj.offsets pointer on Windows VS CI. Go figure.\n+\t */\n+\tswitch (type) {\n+\tcase BLOCK_TYPE_OBJ:\n+\t{\n+\t\tstruct reftable_obj_record obj = { 0 };\n+\t\tclean.u.obj = obj;\n+\t\tbreak;\n+\t}\n+\tcase BLOCK_TYPE_INDEX:\n+\t{\n+\t\tstruct reftable_index_record idx = {\n+\t\t\t.last_key = STRBUF_INIT,\n+\t\t};\n+\t\tclean.u.idx = idx;\n+\t\tbreak;\n+\t}\n+\tcase BLOCK_TYPE_REF:\n+\t{\n+\t\tstruct reftable_ref_record ref = { 0 };\n+\t\tclean.u.ref = ref;\n+\t\tbreak;\n+\t}\n+\tcase BLOCK_TYPE_LOG:\n+\t{\n+\t\tstruct reftable_log_record log = { 0 };\n+\t\tclean.u.log = log;\n+\t\tbreak;\n+\t}\n+\t}\n+\treturn clean;\n+}\ndiff --git a/reftable/record.h b/reftable/record.h\nindex da75d7d1f11..010a322e901 100644\n--- a/reftable/record.h\n+++ b/reftable/record.h\n@@ -63,16 +63,10 @@ struct reftable_record_vtable {\n \tint (*equal)(const void *a, const void *b, int hash_size);\n };\n \n-/* record is a generic wrapper for different types of records. */\n-struct reftable_record {\n-\tvoid *data;\n-\tstruct reftable_record_vtable *ops;\n-};\n-\n /* returns true for recognized block types. Block start with the block type. */\n int reftable_is_block_type(uint8_t typ);\n \n-/* creates a malloced record of the given type. Dispose with record_destroy */\n+/* return an initialized record for the given type */\n struct reftable_record reftable_new_record(uint8_t typ);\n \n /* Encode `key` into `dest`. Sets `is_restart` to indicate a restart. Returns\n@@ -100,6 +94,22 @@ struct reftable_obj_record {\n \tint offset_len;\n };\n \n+/* record is a generic wrapper for different types of records. It is normally\n+ * created on the stack, or embedded within another struct. If the type is\n+ * known, a fresh instance can be initialized explicitly. Otherwise, use\n+ * reftable_new_record() to initialize generically (as the index_record is not\n+ * valid as 0-initialized structure)\n+ */\n+struct reftable_record {\n+\tuint8_t type;\n+\tunion {\n+\t\tstruct reftable_ref_record ref;\n+\t\tstruct reftable_log_record log;\n+\t\tstruct reftable_obj_record obj;\n+\t\tstruct reftable_index_record idx;\n+\t} u;\n+};\n+\n /* see struct record_vtable */\n int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size);\n void reftable_record_key(struct reftable_record *rec, struct strbuf *dest);\n@@ -114,25 +124,9 @@ int reftable_record_decode(struct reftable_record *rec, struct strbuf key,\n \t\t\t int hash_size);\n int reftable_record_is_deletion(struct reftable_record *rec);\n \n-/* zeroes out the embedded record */\n+/* frees and zeroes out the embedded record */\n void reftable_record_release(struct reftable_record *rec);\n \n-/* clear and deallocate embedded record, and zero `rec`. */\n-void reftable_record_destroy(struct reftable_record *rec);\n-\n-/* initialize generic records from concrete records. The generic record should\n- * be zeroed out. */\n-void reftable_record_from_obj(struct reftable_record *rec,\n-\t\t\t struct reftable_obj_record *objrec);\n-void reftable_record_from_index(struct reftable_record *rec,\n-\t\t\t\tstruct reftable_index_record *idxrec);\n-void reftable_record_from_ref(struct reftable_record *rec,\n-\t\t\t struct reftable_ref_record *refrec);\n-void reftable_record_from_log(struct reftable_record *rec,\n-\t\t\t struct reftable_log_record *logrec);\n-struct reftable_ref_record *reftable_record_as_ref(struct reftable_record *ref);\n-struct reftable_log_record *reftable_record_as_log(struct reftable_record *ref);\n-\n /* for qsort. */\n int reftable_ref_record_compare_name(const void *a, const void *b);\n \ndiff --git a/reftable/record_test.c b/reftable/record_test.c\nindex 92680848156..4a025c48104 100644\n--- a/reftable/record_test.c\n+++ b/reftable/record_test.c\n@@ -16,13 +16,14 @@\n \n static void test_copy(struct reftable_record *rec)\n {\n-\tstruct reftable_record copy =\n-\t\treftable_new_record(reftable_record_type(rec));\n+\tstruct reftable_record copy = { 0 };\n+\tuint8_t type = reftable_record_type(rec);\n+\tcopy = reftable_new_record(type);\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \t/* do it twice to catch memory leaks */\n \treftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);\n \tEXPECT(reftable_record_equal(rec, ©, GIT_SHA1_RAWSZ));\n-\treftable_record_destroy(©);\n+\treftable_record_release(©);\n }\n \n static void test_varint_roundtrip(void)\n@@ -95,11 +96,11 @@ static void test_reftable_ref_record_roundtrip(void)\n \tint i = 0;\n \n \tfor (i = REFTABLE_REF_DELETION; i < REFTABLE_NR_REF_VALUETYPES; i++) {\n-\t\tstruct reftable_ref_record in = { NULL };\n-\t\tstruct reftable_ref_record out = { NULL };\n-\t\tstruct reftable_record rec_out = { NULL };\n+\t\tstruct reftable_record in = {\n+\t\t\t.type = BLOCK_TYPE_REF,\n+\t\t};\n+\t\tstruct reftable_record out = { .type = BLOCK_TYPE_REF };\n \t\tstruct strbuf key = STRBUF_INIT;\n-\t\tstruct reftable_record rec = { NULL };\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n \t\t\t.buf = buffer,\n@@ -108,48 +109,46 @@ static void test_reftable_ref_record_roundtrip(void)\n \n \t\tint n, m;\n \n-\t\tin.value_type = i;\n+\t\tin.u.ref.value_type = i;\n \t\tswitch (i) {\n \t\tcase REFTABLE_REF_DELETION:\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_VAL1:\n-\t\t\tin.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val1, 1);\n+\t\t\tin.u.ref.value.val1 = reftable_malloc(GIT_SHA1_RAWSZ);\n+\t\t\tset_hash(in.u.ref.value.val1, 1);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_VAL2:\n-\t\t\tin.value.val2.value = reftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val2.value, 1);\n-\t\t\tin.value.val2.target_value =\n+\t\t\tin.u.ref.value.val2.value =\n+\t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n+\t\t\tset_hash(in.u.ref.value.val2.value, 1);\n+\t\t\tin.u.ref.value.val2.target_value =\n \t\t\t\treftable_malloc(GIT_SHA1_RAWSZ);\n-\t\t\tset_hash(in.value.val2.target_value, 2);\n+\t\t\tset_hash(in.u.ref.value.val2.target_value, 2);\n \t\t\tbreak;\n \t\tcase REFTABLE_REF_SYMREF:\n-\t\t\tin.value.symref = xstrdup(\"target\");\n+\t\t\tin.u.ref.value.symref = xstrdup(\"target\");\n \t\t\tbreak;\n \t\t}\n-\t\tin.refname = xstrdup(\"refs/heads/master\");\n+\t\tin.u.ref.refname = xstrdup(\"refs/heads/master\");\n \n-\t\treftable_record_from_ref(&rec, &in);\n-\t\ttest_copy(&rec);\n+\t\ttest_copy(&in);\n \n-\t\tEXPECT(reftable_record_val_type(&rec) == i);\n+\t\tEXPECT(reftable_record_val_type(&in) == i);\n \n-\t\treftable_record_key(&rec, &key);\n-\t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\t\treftable_record_key(&in, &key);\n+\t\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n > 0);\n \n \t\t/* decode into a non-zero reftable_record to test for leaks. */\n-\n-\t\treftable_record_from_ref(&rec_out, &out);\n-\t\tm = reftable_record_decode(&rec_out, key, i, dest,\n-\t\t\t\t\t GIT_SHA1_RAWSZ);\n+\t\tm = reftable_record_decode(&out, key, i, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_ref_record_equal(&in, &out, GIT_SHA1_RAWSZ));\n-\t\treftable_record_release(&rec_out);\n+\t\tEXPECT(reftable_ref_record_equal(&in.u.ref, &out.u.ref,\n+\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n+\t\treftable_record_release(&in);\n \n \t\tstrbuf_release(&key);\n-\t\treftable_ref_record_release(&in);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n@@ -202,7 +201,7 @@ static void test_reftable_log_record_roundtrip(void)\n \tset_test_hash(in[0].value.update.new_hash, 1);\n \tset_test_hash(in[0].value.update.old_hash, 2);\n \tfor (i = 0; i < ARRAY_SIZE(in); i++) {\n-\t\tstruct reftable_record rec = { NULL };\n+\t\tstruct reftable_record rec = { .type = BLOCK_TYPE_LOG };\n \t\tstruct strbuf key = STRBUF_INIT;\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n@@ -210,23 +209,25 @@ static void test_reftable_log_record_roundtrip(void)\n \t\t\t.len = sizeof(buffer),\n \t\t};\n \t\t/* populate out, to check for leaks. */\n-\t\tstruct reftable_log_record out = {\n-\t\t\t.refname = xstrdup(\"old name\"),\n-\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n-\t\t\t.value = {\n-\t\t\t\t.update = {\n-\t\t\t\t\t.new_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n-\t\t\t\t\t.old_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n-\t\t\t\t\t.name = xstrdup(\"old name\"),\n-\t\t\t\t\t.email = xstrdup(\"old@email\"),\n-\t\t\t\t\t.message = xstrdup(\"old message\"),\n+\t\tstruct reftable_record out = {\n+\t\t\t.type = BLOCK_TYPE_LOG,\n+\t\t\t.u.log = {\n+\t\t\t\t.refname = xstrdup(\"old name\"),\n+\t\t\t\t.value_type = REFTABLE_LOG_UPDATE,\n+\t\t\t\t.value = {\n+\t\t\t\t\t.update = {\n+\t\t\t\t\t\t.new_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t\t.old_hash = reftable_calloc(GIT_SHA1_RAWSZ),\n+\t\t\t\t\t\t.name = xstrdup(\"old name\"),\n+\t\t\t\t\t\t.email = xstrdup(\"old@email\"),\n+\t\t\t\t\t\t.message = xstrdup(\"old message\"),\n+\t\t\t\t\t},\n \t\t\t\t},\n \t\t\t},\n \t\t};\n-\t\tstruct reftable_record rec_out = { NULL };\n \t\tint n, m, valtype;\n \n-\t\treftable_record_from_log(&rec, &in[i]);\n+\t\trec.u.log = in[i];\n \n \t\ttest_copy(&rec);\n \n@@ -234,16 +235,16 @@ static void test_reftable_log_record_roundtrip(void)\n \n \t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n >= 0);\n-\t\treftable_record_from_log(&rec_out, &out);\n \t\tvaltype = reftable_record_val_type(&rec);\n-\t\tm = reftable_record_decode(&rec_out, key, valtype, dest,\n+\t\tm = reftable_record_decode(&out, key, valtype, dest,\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_log_record_equal(&in[i], &out, GIT_SHA1_RAWSZ));\n+\t\tEXPECT(reftable_log_record_equal(&in[i], &out.u.log,\n+\t\t\t\t\t\t GIT_SHA1_RAWSZ));\n \t\treftable_log_record_release(&in[i]);\n \t\tstrbuf_release(&key);\n-\t\treftable_record_release(&rec_out);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n@@ -311,41 +312,43 @@ static void test_reftable_obj_record_roundtrip(void)\n \t\t\t\t\t } };\n \tint i = 0;\n \tfor (i = 0; i < ARRAY_SIZE(recs); i++) {\n-\t\tstruct reftable_obj_record in = recs[i];\n \t\tuint8_t buffer[1024] = { 0 };\n \t\tstruct string_view dest = {\n \t\t\t.buf = buffer,\n \t\t\t.len = sizeof(buffer),\n \t\t};\n-\t\tstruct reftable_record rec = { NULL };\n+\t\tstruct reftable_record in = {\n+\t\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t\t.u.obj = recs[i],\n+\t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n-\t\tstruct reftable_obj_record out = { NULL };\n-\t\tstruct reftable_record rec_out = { NULL };\n+\t\tstruct reftable_record out = { .type = BLOCK_TYPE_OBJ };\n \t\tint n, m;\n \t\tuint8_t extra;\n \n-\t\treftable_record_from_obj(&rec, &in);\n-\t\ttest_copy(&rec);\n-\t\treftable_record_key(&rec, &key);\n-\t\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\t\ttest_copy(&in);\n+\t\treftable_record_key(&in, &key);\n+\t\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \t\tEXPECT(n > 0);\n-\t\textra = reftable_record_val_type(&rec);\n-\t\treftable_record_from_obj(&rec_out, &out);\n-\t\tm = reftable_record_decode(&rec_out, key, extra, dest,\n+\t\textra = reftable_record_val_type(&in);\n+\t\tm = reftable_record_decode(&out, key, extra, dest,\n \t\t\t\t\t GIT_SHA1_RAWSZ);\n \t\tEXPECT(n == m);\n \n-\t\tEXPECT(reftable_record_equal(&rec, &rec_out, GIT_SHA1_RAWSZ));\n+\t\tEXPECT(reftable_record_equal(&in, &out, GIT_SHA1_RAWSZ));\n \t\tstrbuf_release(&key);\n-\t\treftable_record_release(&rec_out);\n+\t\treftable_record_release(&out);\n \t}\n }\n \n static void test_reftable_index_record_roundtrip(void)\n {\n-\tstruct reftable_index_record in = {\n-\t\t.offset = 42,\n-\t\t.last_key = STRBUF_INIT,\n+\tstruct reftable_record in = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = {\n+\t\t\t.offset = 42,\n+\t\t\t.last_key = STRBUF_INIT,\n+\t\t},\n \t};\n \tuint8_t buffer[1024] = { 0 };\n \tstruct string_view dest = {\n@@ -353,31 +356,30 @@ static void test_reftable_index_record_roundtrip(void)\n \t\t.len = sizeof(buffer),\n \t};\n \tstruct strbuf key = STRBUF_INIT;\n-\tstruct reftable_record rec = { NULL };\n-\tstruct reftable_index_record out = { .last_key = STRBUF_INIT };\n-\tstruct reftable_record out_rec = { NULL };\n+\tstruct reftable_record out = {\n+\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t.u.idx = { .last_key = STRBUF_INIT },\n+\t};\n \tint n, m;\n \tuint8_t extra;\n \n-\tstrbuf_addstr(&in.last_key, \"refs/heads/master\");\n-\treftable_record_from_index(&rec, &in);\n-\treftable_record_key(&rec, &key);\n-\ttest_copy(&rec);\n+\tstrbuf_addstr(&in.u.idx.last_key, \"refs/heads/master\");\n+\treftable_record_key(&in, &key);\n+\ttest_copy(&in);\n \n-\tEXPECT(0 == strbuf_cmp(&key, &in.last_key));\n-\tn = reftable_record_encode(&rec, dest, GIT_SHA1_RAWSZ);\n+\tEXPECT(0 == strbuf_cmp(&key, &in.u.idx.last_key));\n+\tn = reftable_record_encode(&in, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(n > 0);\n \n-\textra = reftable_record_val_type(&rec);\n-\treftable_record_from_index(&out_rec, &out);\n-\tm = reftable_record_decode(&out_rec, key, extra, dest, GIT_SHA1_RAWSZ);\n+\textra = reftable_record_val_type(&in);\n+\tm = reftable_record_decode(&out, key, extra, dest, GIT_SHA1_RAWSZ);\n \tEXPECT(m == n);\n \n-\tEXPECT(reftable_record_equal(&rec, &out_rec, GIT_SHA1_RAWSZ));\n+\tEXPECT(reftable_record_equal(&in, &out, GIT_SHA1_RAWSZ));\n \n-\treftable_record_release(&out_rec);\n+\treftable_record_release(&out);\n \tstrbuf_release(&key);\n-\tstrbuf_release(&in.last_key);\n+\tstrbuf_release(&in.u.idx.last_key);\n }\n \n int record_test_main(int argc, const char *argv[])\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex 83a23daf60b..96a5155ea77 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -256,8 +256,10 @@ done:\n int reftable_writer_add_ref(struct reftable_writer *w,\n \t\t\t struct reftable_ref_record *ref)\n {\n-\tstruct reftable_record rec = { NULL };\n-\tstruct reftable_ref_record copy = *ref;\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_REF,\n+\t\t.u.ref = *ref,\n+\t};\n \tint err = 0;\n \n \tif (ref->refname == NULL)\n@@ -266,8 +268,7 @@ int reftable_writer_add_ref(struct reftable_writer *w,\n \t ref->update_index > w->max_update_index)\n \t\treturn REFTABLE_API_ERROR;\n \n-\treftable_record_from_ref(&rec, ©);\n-\tcopy.update_index -= w->min_update_index;\n+\trec.u.ref.update_index -= w->min_update_index;\n \n \terr = writer_add_record(w, &rec);\n \tif (err < 0)\n@@ -306,7 +307,10 @@ int reftable_writer_add_refs(struct reftable_writer *w,\n static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n \t\t\t\t\t struct reftable_log_record *log)\n {\n-\tstruct reftable_record rec = { NULL };\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_LOG,\n+\t\t.u.log = *log,\n+\t};\n \tif (w->block_writer &&\n \t block_writer_type(w->block_writer) == BLOCK_TYPE_REF) {\n \t\tint err = writer_finish_public_section(w);\n@@ -317,7 +321,6 @@ static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n \tw->next -= w->pending_padding;\n \tw->pending_padding = 0;\n \n-\treftable_record_from_log(&rec, log);\n \treturn writer_add_record(w, &rec);\n }\n \n@@ -398,8 +401,10 @@ static int writer_finish_section(struct reftable_writer *w)\n \t\tw->index_len = 0;\n \t\tw->index_cap = 0;\n \t\tfor (i = 0; i < idx_len; i++) {\n-\t\t\tstruct reftable_record rec = { NULL };\n-\t\t\treftable_record_from_index(&rec, idx + i);\n+\t\t\tstruct reftable_record rec = {\n+\t\t\t\t.type = BLOCK_TYPE_INDEX,\n+\t\t\t\t.u.idx = idx[i],\n+\t\t\t};\n \t\t\tif (block_writer_add(w->block_writer, &rec) == 0) {\n \t\t\t\tcontinue;\n \t\t\t}\n@@ -467,17 +472,18 @@ static void write_object_record(void *void_arg, void *key)\n {\n \tstruct write_record_arg *arg = void_arg;\n \tstruct obj_index_tree_node *entry = key;\n-\tstruct reftable_obj_record obj_rec = {\n-\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n-\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n-\t\t.offsets = entry->offsets,\n-\t\t.offset_len = entry->offset_len,\n+\tstruct reftable_record rec = {\n+\t\t.type = BLOCK_TYPE_OBJ,\n+\t\t.u.obj = {\n+\t\t\t.hash_prefix = (uint8_t *)entry->hash.buf,\n+\t\t\t.hash_prefix_len = arg->w->stats.object_id_len,\n+\t\t\t.offsets = entry->offsets,\n+\t\t\t.offset_len = entry->offset_len,\n+\t\t}\n \t};\n-\tstruct reftable_record rec = { NULL };\n \tif (arg->err < 0)\n \t\tgoto done;\n \n-\treftable_record_from_obj(&rec, &obj_rec);\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \tif (arg->err == 0)\n \t\tgoto done;\n@@ -490,7 +496,7 @@ static void write_object_record(void *void_arg, void *key)\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \tif (arg->err == 0)\n \t\tgoto done;\n-\tobj_rec.offset_len = 0;\n+\trec.u.obj.offset_len = 0;\n \targ->err = block_writer_add(arg->w->block_writer, &rec);\n \n \t/* Should be able to write into a fresh block. */\n-- \ngitgitgadget\n\n"},{"id":"446803","messageId":"220124.86mtjlgcl2.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"cc47ec7a50b44e6734a36e1fdb7c21e798efab7e.1643051624.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v7 01/16] reftable: fix OOB stack write in print functions","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-24T19:24:50Z","receivedAt":"2022-01-24T19:59:22Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Jan 24 2022, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> Remove /* BUG */ comment, whose meaning has escaped my memory.\n>\n> Signed-off-by: Han-Wen Nienhuys \n> ---\n> reftable/record.c | 4 ++--\n> 1 file changed, 2 insertions(+), 2 deletions(-)\n>\n> diff --git a/reftable/record.c b/reftable/record.c\n> index 6a5dac32dc6..1f236b47d1c 100644\n> --- a/reftable/record.c\n> +++ b/reftable/record.c\n> @@ -254,7 +254,7 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)\n> void reftable_ref_record_print(struct reftable_ref_record *ref,\n> \t\t\t uint32_t hash_id)\n> {\n> -\tchar hex[2 * GIT_SHA256_RAWSZ + 1] = { 0 }; /* BUG */\n> +\tchar hex[GIT_MAX_HEXSZ + 1];\n> \tprintf(\"ref{%s(%\" PRIu64 \") \", ref->refname, ref->update_index);\n> \tswitch (ref->value_type) {\n> \tcase REFTABLE_REF_SYMREF:\n> @@ -586,7 +586,7 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {\n> void reftable_log_record_print(struct reftable_log_record *log,\n> \t\t\t uint32_t hash_id)\n> {\n> -\tchar hex[GIT_SHA256_RAWSZ + 1] = { 0 };\n> +\tchar hex[GIT_MAX_HEXSZ + 1] = { 0 };\n> \n> \tswitch (log->value_type) {\n> \tcase REFTABLE_LOG_DELETION:\n\nFWIW it seems to originate from\nhttps://github.com/hanwen/reftable/commit/62bbd28 and first appeared\non-list in\nhttps://lore.kernel.org/git/pull.847.v4.git.git.1607522429.gitgitgadget@gmail.com/\n(i.e. any /SHA.*BUG/ in this code). Probably just in reference to having\nto hardcode the \"2 * \" and/or SHA256 which later/now is using a more\ngeneric macro.\n"},{"id":"446807","messageId":"220124.86ee4whpt5.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"pull.1152.v7.git.git.1643051624.gitgitgadget@gmail.com","subject":"Re: [PATCH v7 00/16] Reftable coverity fixes","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-24T19:48:54Z","receivedAt":"2022-01-24T20:54:17Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Jan 24 2022, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> This series was targeted to 'master'.\n>\n> This addresses some complaints from Coverity that Peff reported.\n>\n> v7:\n>\n> * address avarab's comments.\n\nThanks. I just had time to skim it now but this overall LGTM. I noted a\ncompilation error on xlc in another reply that I'm sorry I didn't test\nbefore, but provided a hopefully easy fix for.\n\n> @@ reftable/record.c: static int reftable_ref_record_equal_void(const void *a,\n> +static void reftable_ref_record_print_void(const void *rec,\n> +\t\t\t\t\t int hash_size)\n> +{\n> -+\treftable_ref_record_print_sz((struct reftable_ref_record *) rec, hash_size);\n> ++\treftable_ref_record_print_sz(\n> ++\t\t(struct reftable_ref_record *) rec, hash_size);\n> +}\n> +\n\nJust a nit, and only noting this since you seemed to be adjusting your\nclang-format settings (or so I recall), but I think the more usual\nwrapping in the codebase is:\n \n diff --git a/reftable/record.c b/reftable/record.c\n index 8330b7a2589..ee42897460d 100644\n --- a/reftable/record.c\n +++ b/reftable/record.c\n @@ -464,8 +464,8 @@ static int reftable_ref_record_equal_void(const void *a,\n static void reftable_ref_record_print_void(const void *rec,\n int hash_size)\n {\n - reftable_ref_record_print_sz(\n - (struct reftable_ref_record *) rec, hash_size);\n + reftable_ref_record_print_sz((struct reftable_ref_record *)rec,\n + hash_size);\n }\n \n static struct reftable_record_vtable reftable_ref_record_vtable = {\n \n\nI.e. to \"keep going\" with the initial line until we hit 79 characters,\nand then wrap it. Then for casts \"(x *)y\" rather than \"(x *) y\".\n"},{"id":"446819","messageId":"220124.86ilu8hqag.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"047bbb75fcbb67c41d307bf5fa44587a76cc0777.1643051624.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v7 14/16] reftable: make reftable_record a tagged union","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-24T19:39:20Z","receivedAt":"2022-01-25T02:00:21Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Jan 24 2022, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n> [...]\n> @@ -129,17 +130,25 @@ void reftable_iterator_destroy(struct reftable_iterator *it)\n> int reftable_iterator_next_ref(struct reftable_iterator *it,\n> \t\t\t struct reftable_ref_record *ref)\n> {\n> -\tstruct reftable_record rec = { NULL };\n> -\treftable_record_from_ref(&rec, ref);\n> -\treturn iterator_next(it, &rec);\n> +\tstruct reftable_record rec = {\n> +\t\t.type = BLOCK_TYPE_REF,\n> +\t\t.u.ref = *ref,\n> +\t};\n> +\tint err = iterator_next(it, &rec);\n> +\t*ref = rec.u.ref;\n> +\treturn err;\n> }\n\nI didn't test the v6 on xlc on AIX 7.1, but found that it refuses to\ncompile this code (but the one on AIX 7.2 is OK with it):\n \n \"reftable/generic.c\", line 135.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_ref_record\" is not allowed.\n \"reftable/generic.c\", line 147.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_log_record\" is not allowed.\n \"reftable/writer.c\", line 261.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_ref_record\" is not allowed.\n \"reftable/writer.c\", line 312.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_log_record\" is not allowed.\n \"reftable/writer.c\", line 406.45: 1506-196 (S) Initialization between types \"unsigned long long\" and \"struct reftable_index_record\" is not allowed.\n\nI.e. you're dereferencing a struct type here to get at its first member,\nI'm surprised that gcc and clang don't complain about it.\n\nI think that the below fix-up for this series as a whole (didn't check\nif it was just this patch) should fix it:\n\ndiff --git a/reftable/generic.c b/reftable/generic.c\nindex b176c1cde0a..fe8f060a689 100644\n--- a/reftable/generic.c\n+++ b/reftable/generic.c\n@@ -132,7 +132,7 @@ int reftable_iterator_next_ref(struct reftable_iterator *it,\n {\n \tstruct reftable_record rec = {\n \t\t.type = BLOCK_TYPE_REF,\n-\t\t.u.ref = *ref,\n+\t\t.u.ref = { ref->refname },\n \t};\n \tint err = iterator_next(it, &rec);\n \t*ref = rec.u.ref;\n@@ -144,7 +144,7 @@ int reftable_iterator_next_log(struct reftable_iterator *it,\n {\n \tstruct reftable_record rec = {\n \t\t.type = BLOCK_TYPE_LOG,\n-\t\t.u.log = *log,\n+\t\t.u.log = { log->refname },\n \t};\n \tint err = iterator_next(it, &rec);\n \t*log = rec.u.log;\ndiff --git a/reftable/record_test.c b/reftable/record_test.c\nindex 847abfa2b04..be99de5194b 100644\n--- a/reftable/record_test.c\n+++ b/reftable/record_test.c\n@@ -337,7 +337,7 @@ static void test_reftable_obj_record_roundtrip(void)\n \t\t};\n \t\tstruct reftable_record in = {\n \t\t\t.type = BLOCK_TYPE_OBJ,\n-\t\t\t.u.obj = recs[i],\n+\t\t\t.u.obj = { recs[i].hash_prefix },\n \t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n \t\tstruct reftable_record out = { .type = BLOCK_TYPE_OBJ };\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex 5b768438625..6cc2ef9acaf 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -258,7 +258,7 @@ int reftable_writer_add_ref(struct reftable_writer *w,\n {\n \tstruct reftable_record rec = {\n \t\t.type = BLOCK_TYPE_REF,\n-\t\t.u.ref = *ref,\n+\t\t.u.ref = { ref->refname },\n \t};\n \tint err = 0;\n \n@@ -309,7 +309,7 @@ static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n {\n \tstruct reftable_record rec = {\n \t\t.type = BLOCK_TYPE_LOG,\n-\t\t.u.log = *log,\n+\t\t.u.log = { log->refname },\n \t};\n \tif (w->block_writer &&\n \t block_writer_type(w->block_writer) == BLOCK_TYPE_REF) {\n@@ -403,7 +403,7 @@ static int writer_finish_section(struct reftable_writer *w)\n \t\tfor (i = 0; i < idx_len; i++) {\n \t\t\tstruct reftable_record rec = {\n \t\t\t\t.type = BLOCK_TYPE_INDEX,\n-\t\t\t\t.u.idx = idx[i],\n+\t\t\t\t.u.idx = { idx[i].offset },\n \t\t\t};\n \t\t\tif (block_writer_add(w->block_writer, &rec) == 0) {\n \t\t\t\tcontinue;\n"},{"id":"446822","messageId":"CAFQ2z_M7s9Y_FH5x+qJxyZO0iHq+b_mf8=-dKcmADzFEoGLJUg@mail.gmail.com","threadId":"57047","inReplyTo":"220124.86ilu8hqag.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v7 14/16] reftable: make reftable_record a tagged union","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2022-01-24T19:53:26Z","receivedAt":"2022-01-25T02:01:29Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Mon, Jan 24, 2022 at 8:44 PM Ævar Arnfjörð Bjarmason\n wrote:\n\n> I didn't test the v6 on xlc on AIX 7.1, but found that it refuses to\n> compile this code (but the one on AIX 7.2 is OK with it):\n>\n> \"reftable/generic.c\", line 135.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_ref_record\" is not allowed.\n> \"reftable/generic.c\", line 147.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_log_record\" is not allowed.\n> \"reftable/writer.c\", line 261.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_ref_record\" is not allowed.\n> \"reftable/writer.c\", line 312.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_log_record\" is not allowed.\n> \"reftable/writer.c\", line 406.45: 1506-196 (S) Initialization between types \"unsigned long long\" and \"struct reftable_index_record\" is not allowed.\n>\n\n> I.e. you're dereferencing a struct type here to get at its first member,\n\nNo, that's not what I'm doing.\n\n> > + struct reftable_record rec = {\n> > + .type = BLOCK_TYPE_REF,\n> > + .u.ref = *ref,\n> > + };\n\nreftable_record is tagged union, and u.ref is a reftable_ref_record. I\nintend to shallow-copying the parameter (a reftable_record called\n'ref') into 'rec.u.ref' , precisely as it is written.\n\nDoes this patch pass the unittests?\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\nRegistergericht und -nummer: Hamburg, HRB 86891\nSitz der Gesellschaft: Hamburg\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"446825","messageId":"220124.86a6fkhk8s.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"CAFQ2z_M7s9Y_FH5x+qJxyZO0iHq+b_mf8=-dKcmADzFEoGLJUg@mail.gmail.com","subject":"Re: [PATCH v7 14/16] reftable: make reftable_record a tagged union","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-01-24T21:54:29Z","receivedAt":"2022-01-25T02:08:13Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Jan 24 2022, Han-Wen Nienhuys wrote:\n\n> On Mon, Jan 24, 2022 at 8:44 PM Ævar Arnfjörð Bjarmason\n> wrote:\n>\n>> I didn't test the v6 on xlc on AIX 7.1, but found that it refuses to\n>> compile this code (but the one on AIX 7.2 is OK with it):\n>>\n>> \"reftable/generic.c\", line 135.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_ref_record\" is not allowed.\n>> \"reftable/generic.c\", line 147.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_log_record\" is not allowed.\n>> \"reftable/writer.c\", line 261.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_ref_record\" is not allowed.\n>> \"reftable/writer.c\", line 312.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_log_record\" is not allowed.\n>> \"reftable/writer.c\", line 406.45: 1506-196 (S) Initialization between types \"unsigned long long\" and \"struct reftable_index_record\" is not allowed.\n>>\n>\n>> I.e. you're dereferencing a struct type here to get at its first member,\n>\n> No, that's not what I'm doing.\n>\n>> > + struct reftable_record rec = {\n>> > + .type = BLOCK_TYPE_REF,\n>> > + .u.ref = *ref,\n>> > + };\n>\n> reftable_record is tagged union, and u.ref is a reftable_ref_record. I\n> intend to shallow-copying the parameter (a reftable_record called\n> 'ref') into 'rec.u.ref' , precisely as it is written.\n>\n> Does this patch pass the unittests?\n\nNo, sorry about that. It fails. I didn't have time to run them earlier\n(and it takes a while on that platform).\n\nI don't know how to work around it then...\n"},{"id":"448855","messageId":"220219.86wnhrc8us.gmgdl@evledraar.gmail.com","threadId":"57047","inReplyTo":"220124.86a6fkhk8s.gmgdl@evledraar.gmail.com","subject":"master doesn't compile on xlc 21.01 anymore (old AIX compiler) (was: [PATCH v7 14/16] reftable: make reftable_record a tagged union)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-02-19T12:54:18Z","receivedAt":"2022-02-19T12:57:25Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Jan 24 2022, Ævar Arnfjörð Bjarmason wrote:\n\n> On Mon, Jan 24 2022, Han-Wen Nienhuys wrote:\n>\n>> On Mon, Jan 24, 2022 at 8:44 PM Ævar Arnfjörð Bjarmason\n>> wrote:\n>>\n>>> I didn't test the v6 on xlc on AIX 7.1, but found that it refuses to\n>>> compile this code (but the one on AIX 7.2 is OK with it):\n>>>\n>>> \"reftable/generic.c\", line 135.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_ref_record\" is not allowed.\n>>> \"reftable/generic.c\", line 147.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_log_record\" is not allowed.\n>>> \"reftable/writer.c\", line 261.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_ref_record\" is not allowed.\n>>> \"reftable/writer.c\", line 312.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_log_record\" is not allowed.\n>>> \"reftable/writer.c\", line 406.45: 1506-196 (S) Initialization between types \"unsigned long long\" and \"struct reftable_index_record\" is not allowed.\n>>>\n>>\n>>> I.e. you're dereferencing a struct type here to get at its first member,\n>>\n>> No, that's not what I'm doing.\n>>\n>>> > + struct reftable_record rec = {\n>>> > + .type = BLOCK_TYPE_REF,\n>>> > + .u.ref = *ref,\n>>> > + };\n>>\n>> reftable_record is tagged union, and u.ref is a reftable_ref_record. I\n>> intend to shallow-copying the parameter (a reftable_record called\n>> 'ref') into 'rec.u.ref' , precisely as it is written.\n>>\n>> Does this patch pass the unittests?\n>\n> No, sorry about that. It fails. I didn't have time to run them earlier\n> (and it takes a while on that platform).\n>\n> I don't know how to work around it then...\n\nJust a reminder that now with this landed on master we've hard broken\ncompilation on that version of xlc:\n \n avar@gcc111:[/home/avar]xlc -qversion\n IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72)\n Version: 12.01.0000.0000\n\nThe error is:\n \n \"reftable/generic.c\", line 133.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_ref_record\" is not allowed.\n \"reftable/generic.c\", line 145.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_log_record\" is not allowed.\n\nIs there really no workaround we can think of for this?\n"},{"id":"448856","messageId":"c052b97e-e788-f63a-15c5-b1fbb6a480e8@web.de","threadId":"57047","inReplyTo":"220219.86wnhrc8us.gmgdl@evledraar.gmail.com","subject":"Re: master doesn't compile on xlc 21.01 anymore (old AIX compiler) (was: [PATCH v7 14/16] reftable: make reftable_record a tagged union)","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2022-02-19T15:11:15Z","receivedAt":"2022-02-19T15:11:43Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 19.02.22 um 13:54 schrieb Ævar Arnfjörð Bjarmason:\n>\n> On Mon, Jan 24 2022, Ævar Arnfjörð Bjarmason wrote:\n>\n>> On Mon, Jan 24 2022, Han-Wen Nienhuys wrote:\n>>\n>>> On Mon, Jan 24, 2022 at 8:44 PM Ævar Arnfjörð Bjarmason\n>>> wrote:\n>>>\n>>>> I didn't test the v6 on xlc on AIX 7.1, but found that it refuses to\n>>>> compile this code (but the one on AIX 7.2 is OK with it):\n>>>>\n>>>> \"reftable/generic.c\", line 135.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_ref_record\" is not allowed.\n>>>> \"reftable/generic.c\", line 147.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_log_record\" is not allowed.\n>>>> \"reftable/writer.c\", line 261.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_ref_record\" is not allowed.\n>>>> \"reftable/writer.c\", line 312.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_log_record\" is not allowed.\n>>>> \"reftable/writer.c\", line 406.45: 1506-196 (S) Initialization between types \"unsigned long long\" and \"struct reftable_index_record\" is not allowed.\n>>>>\n>>>\n>>>> I.e. you're dereferencing a struct type here to get at its first member,\n>>>\n>>> No, that's not what I'm doing.\n>>>\n>>>>> + struct reftable_record rec = {\n>>>>> + .type = BLOCK_TYPE_REF,\n>>>>> + .u.ref = *ref,\n>>>>> + };\n>>>\n>>> reftable_record is tagged union, and u.ref is a reftable_ref_record. I\n>>> intend to shallow-copying the parameter (a reftable_record called\n>>> 'ref') into 'rec.u.ref' , precisely as it is written.\n>>>\n>>> Does this patch pass the unittests?\n>>\n>> No, sorry about that. It fails. I didn't have time to run them earlier\n>> (and it takes a while on that platform).\n>>\n>> I don't know how to work around it then...\n>\n> Just a reminder that now with this landed on master we've hard broken\n> compilation on that version of xlc:\n>\n> avar@gcc111:[/home/avar]xlc -qversion\n> IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72)\n> Version: 12.01.0000.0000\n>\n> The error is:\n>\n> \"reftable/generic.c\", line 133.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_ref_record\" is not allowed.\n> \"reftable/generic.c\", line 145.26: 1506-196 (S) Initialization between types \"char*\" and \"struct reftable_log_record\" is not allowed.\n>\n> Is there really no workaround we can think of for this?\n\nThe code is hard to read for me in general due to its many types and\nfunction dispatch tables, but this compiler seems to have an even harder\ntime. Where does it even get that char pointer type from?\n\nCan it compile this?\n\n\tstruct reftable_record rec = {\n\t\t.type = BLOCK_TYPE_REF,\n\t\t.u = {\n\t\t\t.ref = *ref\n\t\t},\n\t};\n\nIf it can't, what about this?\n\n\tstruct reftable_record rec = {\n\t\t.type = BLOCK_TYPE_REF,\n\t};\n\trec.u.ref = *ref;\n\nRené\n"},{"id":"452494","messageId":"patch-1.1-98267695418-20220328T190642Z-avarab@gmail.com","threadId":"57047","inReplyTo":"c052b97e-e788-f63a-15c5-b1fbb6a480e8@web.de","subject":"[PATCH] reftable: make assignments portable to AIX xlc v12.01","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-03-28T19:10:04Z","receivedAt":"2022-03-28T19:10:12Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the assignment syntax introduced in 66c0dabab5e (reftable: make\nreftable_record a tagged union, 2022-01-20) to be portable to AIX xlc\nv12.1:\n\n avar@gcc111:[/home/avar]xlc -qversion\n IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72)\n Version: 12.01.0000.0000\n\nThe error emitted before this was e.g.:\n\n \"reftable/generic.c\", line 133.26: 1506-196 (S) Initialization\n between types \"char*\" and \"struct reftable_ref_record\" is not\n allowed.\n\nThe syntax in the pre-image is supported by e.g. xlc 13.01 on a newer\nAIX version:\n\n avar@gcc119:[/home/avar]xlc -qversion\n IBM XL C/C++ for AIX, V13.1.3 (5725-C72, 5765-J07)\n Version: 13.01.0003.0006\n\nBut as we've otherwise supported this compiler let's not break it\nentirely if it's easy to work around it.\n\nSuggested-by: René Scharfe \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n\nOn Sat, Feb 19 2022, René Scharfe wrote:\n\n> Am 19.02.22 um 13:54 schrieb Ævar Arnfjörð Bjarmason:\n> [...]\n> The code is hard to read for me in general due to its many types and\n> function dispatch tables, but this compiler seems to have an even harder\n> time. Where does it even get that char pointer type from?\n\nSorry about the very late reply, yes thath first suggestion worked,\nd'oh in retrospect. This patch fixes it.\n\n\n reftable/generic.c | 8 ++++++--\n reftable/record_test.c | 4 +++-\n reftable/writer.c | 12 +++++++++---\n 3 files changed, 18 insertions(+), 6 deletions(-)\n\ndiff --git a/reftable/generic.c b/reftable/generic.c\nindex b27d152e89a..57f8032db94 100644\n--- a/reftable/generic.c\n+++ b/reftable/generic.c\n@@ -130,7 +130,9 @@ int reftable_iterator_next_ref(struct reftable_iterator *it,\n {\n \tstruct reftable_record rec = {\n \t\t.type = BLOCK_TYPE_REF,\n-\t\t.u.ref = *ref,\n+\t\t.u = {\n+\t\t\t.ref = *ref\n+\t\t},\n \t};\n \tint err = iterator_next(it, &rec);\n \t*ref = rec.u.ref;\n@@ -142,7 +144,9 @@ int reftable_iterator_next_log(struct reftable_iterator *it,\n {\n \tstruct reftable_record rec = {\n \t\t.type = BLOCK_TYPE_LOG,\n-\t\t.u.log = *log,\n+\t\t.u = {\n+\t\t\t.log = *log,\n+\t\t},\n \t};\n \tint err = iterator_next(it, &rec);\n \t*log = rec.u.log;\ndiff --git a/reftable/record_test.c b/reftable/record_test.c\nindex f91ea5e8830..70ae78feca4 100644\n--- a/reftable/record_test.c\n+++ b/reftable/record_test.c\n@@ -339,7 +339,9 @@ static void test_reftable_obj_record_roundtrip(void)\n \t\t};\n \t\tstruct reftable_record in = {\n \t\t\t.type = BLOCK_TYPE_OBJ,\n-\t\t\t.u.obj = recs[i],\n+\t\t\t.u = {\n+\t\t\t\t.obj = recs[i],\n+\t\t\t},\n \t\t};\n \t\tstruct strbuf key = STRBUF_INIT;\n \t\tstruct reftable_record out = { .type = BLOCK_TYPE_OBJ };\ndiff --git a/reftable/writer.c b/reftable/writer.c\nindex 6d979e245ff..427f1317c6b 100644\n--- a/reftable/writer.c\n+++ b/reftable/writer.c\n@@ -257,7 +257,9 @@ int reftable_writer_add_ref(struct reftable_writer *w,\n {\n \tstruct reftable_record rec = {\n \t\t.type = BLOCK_TYPE_REF,\n-\t\t.u.ref = *ref,\n+\t\t.u = {\n+\t\t\t.ref = *ref\n+\t\t},\n \t};\n \tint err = 0;\n \n@@ -308,7 +310,9 @@ static int reftable_writer_add_log_verbatim(struct reftable_writer *w,\n {\n \tstruct reftable_record rec = {\n \t\t.type = BLOCK_TYPE_LOG,\n-\t\t.u.log = *log,\n+\t\t.u = {\n+\t\t\t.log = *log,\n+\t\t},\n \t};\n \tif (w->block_writer &&\n \t block_writer_type(w->block_writer) == BLOCK_TYPE_REF) {\n@@ -401,7 +405,9 @@ static int writer_finish_section(struct reftable_writer *w)\n \t\tfor (i = 0; i < idx_len; i++) {\n \t\t\tstruct reftable_record rec = {\n \t\t\t\t.type = BLOCK_TYPE_INDEX,\n-\t\t\t\t.u.idx = idx[i],\n+\t\t\t\t.u = {\n+\t\t\t\t\t.idx = idx[i],\n+\t\t\t\t},\n \t\t\t};\n \t\t\tif (block_writer_add(w->block_writer, &rec) == 0) {\n \t\t\t\tcontinue;\n-- \n2.35.1.1548.gaad2ca01774\n\n"},{"id":"452513","messageId":"xmqq4k3h23t7.fsf@gitster.g","threadId":"57047","inReplyTo":"patch-1.1-98267695418-20220328T190642Z-avarab@gmail.com","subject":"Re: [PATCH] reftable: make assignments portable to AIX xlc v12.01","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-03-28T20:57:24Z","receivedAt":"2022-03-28T20:57:28Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> diff --git a/reftable/generic.c b/reftable/generic.c\n> index b27d152e89a..57f8032db94 100644\n> --- a/reftable/generic.c\n> +++ b/reftable/generic.c\n> @@ -130,7 +130,9 @@ int reftable_iterator_next_ref(struct reftable_iterator *it,\n> {\n> \tstruct reftable_record rec = {\n> \t\t.type = BLOCK_TYPE_REF,\n> -\t\t.u.ref = *ref,\n> +\t\t.u = {\n> +\t\t\t.ref = *ref\n> +\t\t},\n> \t};\n\nHaving to do this does *not* look so bad.\n"},{"id":"452558","messageId":"CAFQ2z_PBk57smzqDMVGN7YO3TvutrpaXERfKU4Mp45RfWkwcuQ@mail.gmail.com","threadId":"57047","inReplyTo":"patch-1.1-98267695418-20220328T190642Z-avarab@gmail.com","subject":"Re: [PATCH] reftable: make assignments portable to AIX xlc v12.01","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2022-03-29T12:12:05Z","receivedAt":"2022-03-29T12:12:27Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Mon, Mar 28, 2022 at 9:10 PM Ævar Arnfjörð Bjarmason\n wrote:\n>\n> Change the assignment syntax introduced in 66c0dabab5e (reftable: make\n> reftable_record a tagged union, 2022-01-20) to be portable to AIX xlc\n> v12.1:\n\nLGTM.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n"},{"id":"452577","messageId":"xmqq5ynwyck5.fsf@gitster.g","threadId":"57047","inReplyTo":"CAFQ2z_PBk57smzqDMVGN7YO3TvutrpaXERfKU4Mp45RfWkwcuQ@mail.gmail.com","subject":"Re: [PATCH] reftable: make assignments portable to AIX xlc v12.01","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-03-29T15:59:38Z","receivedAt":"2022-03-29T15:59:46Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Han-Wen Nienhuys writes:\n\n> On Mon, Mar 28, 2022 at 9:10 PM Ævar Arnfjörð Bjarmason\n> wrote:\n>>\n>> Change the assignment syntax introduced in 66c0dabab5e (reftable: make\n>> reftable_record a tagged union, 2022-01-20) to be portable to AIX xlc\n>> v12.1:\n>\n> LGTM.\n\nThanks; queued.\n"}]}