git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 4/4] object-file: create ODB transaction temporary directory eagerly

From
Karthik Nayak <karthik.188@gmail.com>
Date
Nov 11, 2026, 16:48 UTC
Message-ID
<CAOLa=ZShhVC=7uYs4_CP6pUSuiYziQrbN8EbJhpqYToixPV-VA@mail.gmail.com>
In-Reply-To
<20261110002251.2007617-5-jltobler@gmail.com>
Justin Tobler <jltobler@gmail.com> writes:
Show 37 quoted lines
> By default, ODB transactions for the "files" backend create their
> temporary object directory lazily once the first loose object is about
> to be written, and only if batched fsync is enabled. Transactions
> configured for git-receive-pack(1) are the exception as the directory is
> needed to quarantine incoming objects up front. Lazily creating the
> temporary directory in this manner is awkward for a couple of reasons:
>
>   - Writing loose objects has to know about the transaction internals to
>     trigger the lazy creation of the temporary object directory.
>
>   - Whether objects are staged in a temporary object directory, and thus
>     whether they become visible at once on commit, depends on the
>     configured fsync method.
>
> Create the temporary object directory eagerly for every transaction when
> it is started instead. This also ensures that no object is written to
> the main object database before the transaction is committed.
>
> Note that committing a transaction migrates every written object into
> the primary object directory even when batched fsync is disabled. This
> adds some minor overhead to ODB transaction callers that previously only
> used a temporary directory if batched fsync was enabled. The following
> benchmark uses the default fsync configuration and adds 10000 files of 4
> KiB each to a freshly initialized repository via git-add(1):
>
>   Benchmark 1: git add . (rev = HEAD~)
>     Time (mean ± σ):     895.0 ms ±  23.8 ms    [User: 545.3 ms, System: 340.7 ms]
>     Range (min … max):   866.7 ms … 928.7 ms    20 runs
>
>   Benchmark 2: git add . (rev = HEAD)
>     Time (mean ± σ):     956.3 ms ±  26.1 ms    [User: 541.4 ms, System: 405.2 ms]
>     Range (min … max):   923.9 ms … 993.6 ms    20 runs
>
>   Summary
>     git add . (rev = HEAD~) ran
>       1.07 ± 0.04 times faster than git add . (rev = HEAD)
>

The explanation hre is well written and nicely explains what the cover letter didn't _cover_. Nice.

> Typically large numbers of loose objects are not expected to be written
> into an ODB transaction so the overhead here should be marginal.
>
Question: why is that?
Show 7 quoted lines
> Signed-off-by: Justin Tobler <jltobler@gmail.com>
> ---
>  object-file.c         | 56 +++++++++----------------------------------
>  object-file.h         |  1 -
>  odb/source-loose.c    |  6 -----
>  t/t0004-unwritable.sh | 19 +++++++--------
>  4 files changed, 19 insertions(+), 63 deletions(-)
The changes itself looks good.
Previous: Justin ToblerNext: Justin Tobler
Message 17 of 26 in “object-file: eagerly create ODB transaction temporary directory”
  1. 0/4 object-file: eagerly create ODB transaction temporary directoryJustin Tobler, Nov 10, 2026
  2. 1/4 object-file: rename default transaction prefixJustin Tobler, Nov 10, 2026
  3. Patrick SteinhardtNov 11, 2026
  4. Karthik NayakNov 11, 2026
  5. 2/4 object-file: only flush batched fsync writes when enabledJustin Tobler, Nov 10, 2026
  6. Patrick SteinhardtNov 11, 2026
  7. Justin ToblerNov 17, 2026
  8. Karthik NayakNov 11, 2026
  9. Justin ToblerNov 17, 2026
  10. 3/4 tmp-objdir: preserve errno when directory creation failsJustin Tobler, Nov 10, 2026
  11. Patrick SteinhardtNov 11, 2026
  12. Karthik NayakNov 11, 2026
  13. Justin ToblerNov 17, 2026
  14. 4/4 object-file: create ODB transaction temporary directory eagerlyJustin Tobler, Nov 10, 2026
  15. Patrick SteinhardtNov 11, 2026
  16. Justin ToblerNov 17, 2026
  17. Karthik NayakNov 11, 2026
  18. Justin ToblerNov 17, 2026
  19. Karthik NayakNov 11, 2026
  20. 0/6 object-file: eagerly create ODB transaction temporary directoryJustin Tobler, Nov 19, 2026
  21. 1/6 object-file: rename default transaction prefixJustin Tobler, Nov 19, 2026
  22. 2/6 object-file: only flush batched fsync writes when enabledJustin Tobler, Nov 19, 2026
  23. 3/6 tmp-objdir: inline `setup_tmp_objdir()`Justin Tobler, Nov 19, 2026
  24. 4/6 tmp-objdir: preserve errno when directory creation failsJustin Tobler, Nov 19, 2026
  25. 5/6 odb/transaction: avoid "ODB" in error messagesJustin Tobler, Nov 19, 2026
  26. 6/6 object-file: create ODB transaction temporary directory eagerlyJustin Tobler, Nov 19, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.