git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 2/4] object-file: only flush batched fsync writes when enabled

From
Justin Tobler <jltobler@gmail.com>
Date
Nov 17, 2026, 23:35 UTC
Message-ID
<avzghl3FBW3TTWhV@denethor>
In-Reply-To
<avRqlHdN83O67HB1@pks.im>
On 26/11/11 02:16PM, Patrick Steinhardt wrote:
Show 11 quoted lines
> On Mon, Nov 09, 2026 at 06:22:49PM -0600, Justin Tobler wrote:
> > When committing an ODB transaction for the "files" backend, a full
> > hardware flush is issued against a temporary file before the staged
> > objects are migrated into the main object database. This flush acts as a
> > barrier for objects that have been written with batched fsync, where
> > `odb_transaction_files_fsync()` only issues a writeout request for each
> > object instead of fully syncing it.
> 
> Hmm, okay. The difference here is bascially that we don't issue a
> hardware flush. So we know that the data is not in the page cache
> anymore, but it may still only be cached by the disk.
Performing the hardware sync here is really only intended for batch fsync 

Right, that's what writeout-only gives us for each object with batched fsync, and why the single hardware flush at commit time is needed as a barrier.

That state only arises with batched fsync though. Otherwise, objects are either fully synced individually, in which case they are already durable and the extra flush is redundant, or not synced at all, in which case there is nothing to make durable. So the flush is only useful if there were writeout-only syncs.

Show 10 quoted lines
> > The flush is issued whenever the transaction has a temporary object
> > directory though, regardless of whether batched fsync is actually
> > enabled. For transactions configured for git-receive-pack(1), the
> > temporary object directory is always created, so the flush is also
> > performed when objects are synced individually or not synced at all. In
> > those cases there are no pending writeout requests and the flush is
> > unnecessary. Only issue the flush when batched fsync is enabled
> > accordingly.
> 
> So this is essentially a performance optimization?

Yes and I also think its not really the correct behavior. Even is fsync is disabled, the fsync here happens. It doesn't really harm anything, but I think it makes sense to address as we begin to roll out transactions more widely.

[snip]
> I assume that we only support batched fsyncs for loose objects in the
> first place? In other words, if a transaction were to write a bunch of
> packfiles, it would not be possible to batch-fsync those right now?
Correct, this mechanism was intended to batch fsync loose objects.
> If so this new condition correct to me. But I think this should be
> documented as part of the commit message, because it's an implicit
> assumption.
Makes sense. I'll update the commit message in the next version.

Thanks, -Justin

Previous: Patrick SteinhardtNext: Karthik Nayak
Message 7 of 29 in “object-file: eagerly create ODB transaction temporary directory”
  1. 0/4 object-file: eagerly create ODB transaction temporary directoryJustin Tobler, Nov 10, 2026
  2. 1/4 object-file: rename default transaction prefixJustin Tobler, Nov 10, 2026
  3. Patrick SteinhardtNov 11, 2026
  4. Karthik NayakNov 11, 2026
  5. 2/4 object-file: only flush batched fsync writes when enabledJustin Tobler, Nov 10, 2026
  6. Patrick SteinhardtNov 11, 2026
  7. Justin ToblerNov 17, 2026
  8. Karthik NayakNov 11, 2026
  9. Justin ToblerNov 17, 2026
  10. 3/4 tmp-objdir: preserve errno when directory creation failsJustin Tobler, Nov 10, 2026
  11. Patrick SteinhardtNov 11, 2026
  12. Karthik NayakNov 11, 2026
  13. Justin ToblerNov 17, 2026
  14. 4/4 object-file: create ODB transaction temporary directory eagerlyJustin Tobler, Nov 10, 2026
  15. Patrick SteinhardtNov 11, 2026
  16. Justin ToblerNov 17, 2026
  17. Karthik NayakNov 11, 2026
  18. Justin ToblerNov 17, 2026
  19. Karthik NayakNov 11, 2026
  20. 0/6 object-file: eagerly create ODB transaction temporary directoryJustin Tobler, Nov 19, 2026
  21. 1/6 object-file: rename default transaction prefixJustin Tobler, Nov 19, 2026
  22. 2/6 object-file: only flush batched fsync writes when enabledJustin Tobler, Nov 19, 2026
  23. 3/6 tmp-objdir: inline `setup_tmp_objdir()`Justin Tobler, Nov 19, 2026
  24. 4/6 tmp-objdir: preserve errno when directory creation failsJustin Tobler, Nov 19, 2026
  25. 5/6 odb/transaction: avoid "ODB" in error messagesJustin Tobler, Nov 19, 2026
  26. Patrick SteinhardtNov 19, 2026
  27. 6/6 object-file: create ODB transaction temporary directory eagerlyJustin Tobler, Nov 19, 2026
  28. Patrick SteinhardtNov 19, 2026
  29. Junio C HamanoNov 19, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.