[PATCH v3 00/11] receive-pack: use ODB transactions to stage object writes
- From
Justin Tobler <jltobler@gmail.com>
- Date
- Jul 8, 2026, 23:59 UTC
- Message-ID
- <20260708235925.3992097-1-jltobler@gmail.com>
- In-Reply-To
- <20260708041412.1157499-1-jltobler@gmail.com>
Greetings,
This patch series replaces direct usage of the `tmp_objdir` interfaces in git-receive-pack(1) to instead use the `odb_transaction` interfaces to create/manage a staging area to write objects to. The purpose of this change is to get git-receive-pack(1) one step closer to being ODB backend agnostic. For now, the object writes themselves are still "files" backend specific due to being handled by the git-index-pack(1) and git-unpack-objects(1) child processes. This will be tackled in a separate series though.
Changes since V2:
- Clarified commit log reasoning for embedding
`flush_loose_object_transaction()` logic in commit function.
- Started printed some error messages on transaction errors.
- Removed include statement.
- Fixed transaction leak on `odb_transaction_commit()` error.Changes since V1:
- Adapted other "file" ODB transaction helpers to be more consistent
with current naming scheme.
- Removed redundant NULL transaction handling from
`odb_transaction_files_begin()`.
- `odb_transaction_begin()` now returns an error if there is already
an inflight transaction pending instead of setting the `out` pointer
to NULL.
- Updated `odb_transaction_env()` to return an error code and append
environment variables to a strvec provided as an argument.
- Removed redundant setting of tmpdir environment variables for child
processes after tmpdir has been migrated.
- Split changes adding ODB transaction flags into a separate commit.
- Consistently wire the ODB transaction throughout git-receive-pack
code instead of reading it from `the_repository`.
- Updated user facing error message.
- Updated some comments to better document functions/flags.
- Clarified some commit messages.
- Fixed typos.Thanks, -Justin
Justin Tobler (11): object-file: rename files transaction prepare function object-file: rename files transaction fsync function object-file: embed transaction flush logic in commit function object-file: drop check for inflight transactions object-file: propagate files transaction errors odb/transaction: propagate begin errors odb/transaction: propagate commit errors odb/transaction: add transaction env interface odb/transaction: introduce ODB transaction flags builtin/receive-pack: drop redundant tmpdir env builtin/receive-pack: stage incoming objects via ODB transactions
builtin/add.c | 2 +- builtin/receive-pack.c | 69 ++++++++--------- builtin/unpack-objects.c | 2 +- builtin/update-index.c | 2 +- cache-tree.c | 7 +- object-file.c | 159 +++++++++++++++++++++++++-------------- object-file.h | 8 +- odb/source-files.c | 9 +-- odb/source-inmemory.c | 3 +- odb/source-loose.c | 3 +- odb/source.h | 9 ++- odb/transaction.c | 32 ++++++-- odb/transaction.h | 59 ++++++++++++--- read-cache.c | 7 +- 14 files changed, 241 insertions(+), 130 deletions(-)
Range-diff against v2:
1: 9c14b219ad = 1: 9c14b219ad object-file: rename files transaction prepare function
2: 5703a9e93b = 2: 5703a9e93b object-file: rename files transaction fsync function
3: 4c37398ac8 ! 3: 76204847f2 object-file: embed transaction flush logic in commit function
@@ Commit message
When a "files" transaction is committed,
`flush_loose_object_transaction()` is invoked to handle performing a
hardware flush along with migrating the temporary object directory into
- the primary. In a subsequent commit, the temporary directory is also
- used to write packfiles.
+ the primary and configuring the repository ODB source accordingly. The
+ function name here is a bit misleading because the helper is doing a bit
+ more than just "flushing" the transaction contents. Also, in a
+ subsequent commit, the transaction temporary directory is used to stage
+ packfiles and not just loose objects anymore.
- Instead of maintaining a separate helper function, embed the logic to
- flush and migrate the temporary directory directly into
- `odb_transaction_files_commit()`.
+ Lift the helper function logic directly into
+ `odb_transaction_files_commit()` to more accurately signal to readers
+ the operation being performed.
Signed-off-by: Justin Tobler <jltobler@gmail.com>
4: 623c6b02ea = 4: c97eb7763f object-file: drop check for inflight transactions
5: ca59176657 ! 5: 1f3a1f7714 object-file: propagate files transaction errors
@@ object-file.c: static void odb_transaction_files_prepare(struct odb_transaction
- if (transaction->objdir)
- tmp_objdir_replace_primary_odb(transaction->objdir, 0);
+ if (!transaction->objdir)
-+ return -1;
++ return error(_("unable to create temporary object directory"));
+
+ tmp_objdir_replace_primary_odb(transaction->objdir, 0);
+
@@ object-file.c: static void odb_transaction_files_commit(struct odb_transaction *
*/
- tmp_objdir_migrate(transaction->objdir);
+ if (tmp_objdir_migrate(transaction->objdir))
-+ return -1;
++ return error(_("unable to migrate temporary objects"));
+
transaction->objdir = NULL;
}
6: 717a1ce9a7 ! 6: 09d13272d5 odb/transaction: propagate begin errors
@@ object-file.c: int index_fd(struct index_state *istate, struct object_id *oid,
## odb/transaction.c ##
@@
+ #include "git-compat-util.h"
++#include "gettext.h"
#include "odb/source.h"
#include "odb/transaction.h"
@@ odb/transaction.c
+
if (odb->transaction)
- return NULL;
-+ return -1;
++ return error(_("object database transaction already pending"));
- odb_source_begin_transaction(odb->sources, &odb->transaction);
+ ret = odb_source_begin_transaction(odb->sources, out);
@@ odb/transaction.h
#ifndef ODB_TRANSACTION_H
#define ODB_TRANSACTION_H
-+#include "git-compat-util.h"
+#include "gettext.h"
#include "odb.h"
#include "odb/source.h"
7: ff8e133965 ! 7: 12833d6773 odb/transaction: propagate commit errors
@@ odb/transaction.c: int odb_transaction_begin(struct object_database *odb,
- transaction->commit(transaction);
+ ret = transaction->commit(transaction);
-+ if (ret)
-+ return ret;
-+
transaction->source->odb->transaction = NULL;
free(transaction);
+
-+ return 0;
++ return ret;
}
int odb_transaction_write_object_stream(struct odb_transaction *transaction,
8: 264ba94b83 = 8: f2586f2f34 odb/transaction: add transaction env interface
9: 1e0a491ef2 ! 9: 9d082b5e47 odb/transaction: introduce ODB transaction flags
@@ object-file.c: static int odb_transaction_files_prepare(struct odb_transaction *
- transaction->objdir = tmp_objdir_create(base->source->odb->repo, "bulk-fsync");
+ transaction->objdir = tmp_objdir_create(base->source->odb->repo, transaction->prefix);
if (!transaction->objdir)
- return -1;
+ return error(_("unable to create temporary object directory"));
@@ object-file.c: int index_fd(struct index_state *istate, struct object_id *oid,
int inflight = !!transaction;
@@ odb/transaction.c
int ret;
if (odb->transaction)
- return -1;
+ return error(_("object database transaction already pending"));
- ret = odb_source_begin_transaction(odb->sources, out);
+ ret = odb_source_begin_transaction(odb->sources, out, flags);
@@ odb/transaction.c
## odb/transaction.h ##
@@
- #include "git-compat-util.h"
+
#include "gettext.h"
#include "odb.h"
-#include "odb/source.h"
10: 6c8d878349 = 10: e11d8a6676 builtin/receive-pack: drop redundant tmpdir env
11: 8db95fef56 = 11: fee57c2817 builtin/receive-pack: stage incoming objects via ODB transactionsbase-commit: ab776a62a78576513ee121424adb19597fbb7613
-- 2.55.0.122.gf85a7e6620