Re: [PATCH] reftable: honor core.fsync

From: Patrick Steinhardt <ps@pks.im>
Date: Jan 29, 2024, 09:48 UTC
Message-ID: <Zbd0i9nOeWWNQ2EW@tanuki>
In-Reply-To: <pull.1654.git.git.1706035870956.gitgitgadget@gmail.com>

On Tue, Jan 23, 2024 at 06:51:10PM +0000, John Cai via GitGitGadget wrote:

Show 21 quoted lines

> From: John Cai <johncai86@gmail.com>
> 
> While the reffiles backend honors configured fsync settings, the
> reftable backend does not. Address this by fsyncing reftable files using
> the write-or-die api's fsync_component() in two places: when we
> add additional entries into the table, and when we close the reftable
> writer.
> 
> This commits adds a flush function pointer as a new member of
> reftable_writer because we are not sure that the first argument to the
> *write function pointer always contains a file descriptor. In the case of
> strbuf_add_void, the first argument is a buffer. This way, we can pass
> in a corresponding flush function that knows how to flush depending on
> which writer is being used.
> 
> This patch does not contain tests as they will need to wait for another
> patch to start to exercise the reftable backend. At that point, the
> tests will be added to observe that fsyncs are happening when the
> reftable is in use.
> 
> Signed-off-by: John Cai <johncai86@gmail.com>

I noticed that we missed syncing the "tables.list" file when performing auto-compaction. The below patch is needed on top of what we already have.

The topic is currently in `next`, but not yet in `master`, so we might still squash it in. Junio, please let me know whether you want to do so or whether I shall send this fix-up as a new patch. Thanks!

Patrick

diff --git a/reftable/stack.c b/reftable/stack.c
index ab295341cc..b17cfb9516 100644
--- a/reftable/stack.c
+++ b/reftable/stack.c
@@ -1018,6 +1018,10 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,
 		unlink(new_table_path.buf);
 		goto done;
 	}
+
+	fsync_component_or_die(FSYNC_COMPONENT_REFERENCE, lock_file_fd,
+			       lock_file_name.buf);
+
 	err = close(lock_file_fd);
 	lock_file_fd = -1;
 	if (err < 0) {