threads / patch / 60814

patchreftable/stack: fsync "tables.list" during compaction

Subject: [PATCH] reftable/stack: fsync "tables.list" during compaction

## tl;dr

3 messages between Jan 30, 2024 and Jan 31, 2024. Diffs are folded; open one to read it.

replies: 2people: 2as markdown or json

Patrick Steinhardt· Jan 30, 2024, 05:22 UTC · lore

In 1df18a1c9a (reftable: honor core.fsync, 2024-01-23), we have added code to fsync both newly written reftables as well as "tables.list" to disk. But there are two code paths where "tables.list" is being written:

  - When appending a new table due to a normal ref update.

  - When compacting a range of tables during compaction.

We have only addressed the former code path, but do not yet sync the new "tables.list" file in the latter. Fix this ommission.

Note that we are not yet adding any tests. These tests will be added once the "reftable" backend has been upstreamed.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 reftable/stack.c | 8 ++++++++
 1 file changed, 8 insertions(+)

Show changes to reftable/stack.c +8 −0

diff --git a/reftable/stack.c b/reftable/stack.c
index ab295341cc..01d05933f6 100644
--- a/reftable/stack.c
+++ b/reftable/stack.c
@@ -1018,6 +1018,14 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,
 		unlink(new_table_path.buf);
 		goto done;
 	}
+
+	err = fsync_component(FSYNC_COMPONENT_REFERENCE, lock_file_fd);
+	if (err < 0) {
+		err = REFTABLE_IO_ERROR;
+		unlink(new_table_path.buf);
+		goto done;
+	}
+
 	err = close(lock_file_fd);
 	lock_file_fd = -1;
 	if (err < 0) {

-- 
2.43.GIT

Justin Tobler· Jan 30, 2024, 19:04 UTC · re: Patrick Steinhardt · lore

Re: [PATCH] reftable/stack: fsync "tables.list" during compaction

On Mon, Jan 29, 2024 at 11:23 PM Patrick Steinhardt <ps@pks.im> wrote:

Show 11 quoted lines

>
> In 1df18a1c9a (reftable: honor core.fsync, 2024-01-23), we have added
> code to fsync both newly written reftables as well as "tables.list" to
> disk. But there are two code paths where "tables.list" is being written:
>
>   - When appending a new table due to a normal ref update.
>
>   - When compacting a range of tables during compaction.
>
> We have only addressed the former code path, but do not yet sync the new
> "tables.list" file in the latter. Fix this ommission.

nit: s/ommission/omission

>
> Note that we are not yet adding any tests. These tests will be added
> once the "reftable" backend has been upstreamed.
>

Nice catch! I noticed a small typo in the commit message but otherwise looks good to me.

-Justin

Patrick Steinhardt· Jan 31, 2024, 05:46 UTC · re: Justin Tobler · lore

Re: [PATCH] reftable/stack: fsync "tables.list" during compaction

On Tue, Jan 30, 2024 at 01:04:46PM -0600, Justin Tobler wrote:

Show 14 quoted lines

> On Mon, Jan 29, 2024 at 11:23 PM Patrick Steinhardt <ps@pks.im> wrote:
> >
> > In 1df18a1c9a (reftable: honor core.fsync, 2024-01-23), we have added
> > code to fsync both newly written reftables as well as "tables.list" to
> > disk. But there are two code paths where "tables.list" is being written:
> >
> >   - When appending a new table due to a normal ref update.
> >
> >   - When compacting a range of tables during compaction.
> >
> > We have only addressed the former code path, but do not yet sync the new
> > "tables.list" file in the latter. Fix this ommission.
> 
> nit: s/ommission/omission

I knew this looked weird when writing it! Should've looked it up.

Show 6 quoted lines

> > Note that we are not yet adding any tests. These tests will be added
> > once the "reftable" backend has been upstreamed.
> >
> 
> Nice catch! I noticed a small typo in the commit message but otherwise looks
> good to me.

Thanks. I'll refrain from sending out a new version for this typo alone, but will fix it if other feedback requires a second iteration.

Patrick