Re: [PATCH 2/4] object-file: only flush batched fsync writes when enabled
Justin Tobler <jltobler@gmail.com> writes:
Show 7 quoted lines
> When committing an ODB transaction for the "files" backend, a full
> hardware flush is issued against a temporary file before the staged
> objects are migrated into the main object database. This flush acts as a
> barrier for objects that have been written with batched fsync, where
> `odb_transaction_files_fsync()` only issues a writeout request for each
> object instead of fully syncing it.
>
Does `fsync(fd)` even guarantee syncing of previous writes?
Show 36 quoted lines
> The flush is issued whenever the transaction has a temporary object
> directory though, regardless of whether batched fsync is actually
> enabled. For transactions configured for git-receive-pack(1), the
> temporary object directory is always created, so the flush is also
> performed when objects are synced individually or not synced at all. In
> those cases there are no pending writeout requests and the flush is
> unnecessary. Only issue the flush when batched fsync is enabled
> accordingly.
>
> Signed-off-by: Justin Tobler <jltobler@gmail.com>
> ---
> object-file.c | 43 +++++++++++++++++++++++++------------------
> 1 file changed, 25 insertions(+), 18 deletions(-)
>
> diff --git a/object-file.c b/object-file.c
> index 201d9d6490..da189111bd 100644
> --- a/object-file.c
> +++ b/object-file.c
> @@ -1330,6 +1330,29 @@ static int write_pack_lockfiles(struct odb_transaction_files *transaction)
> return ret;
> }
>
> +static void flush_batched_fsync(struct repository *repo)
> +{
> + struct strbuf temp_path = STRBUF_INIT;
> + struct tempfile *temp;
> +
> + /*
> + * Issue a full hardware flush against a temporary file to ensure
> + * that all objects are durable before any renames occur. The code in
> + * odb_transaction_files_fsync has already issued a writeout
> + * request, but it has not flushed any writeback cache in the storage
> + * hardware or any filesystem logs. This fsync call acts as a barrier
> + * to ensure that the data in each new object file is durable before
> + * the final name is visible.
> + */Nit: This could become a function comment, since it now describes the
function well.
Show 9 quoted lines
> + strbuf_addf(&temp_path, "%s/bulk_fsync_XXXXXX",
> + repo_get_object_directory(repo));
> + temp = xmks_tempfile(temp_path.buf);
> + fsync_or_die(get_tempfile_fd(temp), get_tempfile_path(temp));
> +
> + strbuf_release(&temp_path);
> + delete_tempfile(&temp);
> +}
> +