git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 2/4] object-file: only flush batched fsync writes when enabled

From
Karthik Nayak <karthik.188@gmail.com>
Date
Nov 11, 2026, 14:48 UTC
Message-ID
<CAOLa=ZRp=JeAGhN3m6-RCos3F82__6Wzt2UsKvUjqHSHUg6o_Q@mail.gmail.com>
In-Reply-To
<20261110002251.2007617-3-jltobler@gmail.com>
Justin Tobler <jltobler@gmail.com> writes:
Show 7 quoted lines
> When committing an ODB transaction for the "files" backend, a full
> hardware flush is issued against a temporary file before the staged
> objects are migrated into the main object database. This flush acts as a
> barrier for objects that have been written with batched fsync, where
> `odb_transaction_files_fsync()` only issues a writeout request for each
> object instead of fully syncing it.
>
Does `fsync(fd)` even guarantee syncing of previous writes?
Show 36 quoted lines
> The flush is issued whenever the transaction has a temporary object
> directory though, regardless of whether batched fsync is actually
> enabled. For transactions configured for git-receive-pack(1), the
> temporary object directory is always created, so the flush is also
> performed when objects are synced individually or not synced at all. In
> those cases there are no pending writeout requests and the flush is
> unnecessary. Only issue the flush when batched fsync is enabled
> accordingly.
>
> Signed-off-by: Justin Tobler <jltobler@gmail.com>
> ---
>  object-file.c | 43 +++++++++++++++++++++++++------------------
>  1 file changed, 25 insertions(+), 18 deletions(-)
>
> diff --git a/object-file.c b/object-file.c
> index 201d9d6490..da189111bd 100644
> --- a/object-file.c
> +++ b/object-file.c
> @@ -1330,6 +1330,29 @@ static int write_pack_lockfiles(struct odb_transaction_files *transaction)
>  	return ret;
>  }
>
> +static void flush_batched_fsync(struct repository *repo)
> +{
> +	struct strbuf temp_path = STRBUF_INIT;
> +	struct tempfile *temp;
> +
> +	/*
> +	 * Issue a full hardware flush against a temporary file to ensure
> +	 * that all objects are durable before any renames occur. The code in
> +	 * odb_transaction_files_fsync has already issued a writeout
> +	 * request, but it has not flushed any writeback cache in the storage
> +	 * hardware or any filesystem logs. This fsync call acts as a barrier
> +	 * to ensure that the data in each new object file is durable before
> +	 * the final name is visible.
> +	 */
Nit: This could become a function comment, since it now describes the
function well.
Show 9 quoted lines
> +	strbuf_addf(&temp_path, "%s/bulk_fsync_XXXXXX",
> +		    repo_get_object_directory(repo));
> +	temp = xmks_tempfile(temp_path.buf);
> +	fsync_or_die(get_tempfile_fd(temp), get_tempfile_path(temp));
> +
> +	strbuf_release(&temp_path);
> +	delete_tempfile(&temp);
> +}
> +
[snip]
Previous: Justin ToblerNext: Justin Tobler
Message 8 of 26 in “object-file: eagerly create ODB transaction temporary directory”
  1. 0/4 object-file: eagerly create ODB transaction temporary directoryJustin Tobler, Nov 10, 2026
  2. 1/4 object-file: rename default transaction prefixJustin Tobler, Nov 10, 2026
  3. Patrick SteinhardtNov 11, 2026
  4. Karthik NayakNov 11, 2026
  5. 2/4 object-file: only flush batched fsync writes when enabledJustin Tobler, Nov 10, 2026
  6. Patrick SteinhardtNov 11, 2026
  7. Justin ToblerNov 17, 2026
  8. Karthik NayakNov 11, 2026
  9. Justin ToblerNov 17, 2026
  10. 3/4 tmp-objdir: preserve errno when directory creation failsJustin Tobler, Nov 10, 2026
  11. Patrick SteinhardtNov 11, 2026
  12. Karthik NayakNov 11, 2026
  13. Justin ToblerNov 17, 2026
  14. 4/4 object-file: create ODB transaction temporary directory eagerlyJustin Tobler, Nov 10, 2026
  15. Patrick SteinhardtNov 11, 2026
  16. Justin ToblerNov 17, 2026
  17. Karthik NayakNov 11, 2026
  18. Justin ToblerNov 17, 2026
  19. Karthik NayakNov 11, 2026
  20. 0/6 object-file: eagerly create ODB transaction temporary directoryJustin Tobler, Nov 19, 2026
  21. 1/6 object-file: rename default transaction prefixJustin Tobler, Nov 19, 2026
  22. 2/6 object-file: only flush batched fsync writes when enabledJustin Tobler, Nov 19, 2026
  23. 3/6 tmp-objdir: inline `setup_tmp_objdir()`Justin Tobler, Nov 19, 2026
  24. 4/6 tmp-objdir: preserve errno when directory creation failsJustin Tobler, Nov 19, 2026
  25. 5/6 odb/transaction: avoid "ODB" in error messagesJustin Tobler, Nov 19, 2026
  26. 6/6 object-file: create ODB transaction temporary directory eagerlyJustin Tobler, Nov 19, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.