From: Justin Tobler Date: Wed, 08 Jul 2026 23:59:14 GMT Subject: [PATCH v3 00/11] receive-pack: use ODB transactions to stage object writes Message-ID: <20260708235925.3992097-1-jltobler@gmail.com> In-Reply-To: <20260708041412.1157499-1-jltobler@gmail.com> Greetings, This patch series replaces direct usage of the `tmp_objdir` interfaces in git-receive-pack(1) to instead use the `odb_transaction` interfaces to create/manage a staging area to write objects to. The purpose of this change is to get git-receive-pack(1) one step closer to being ODB backend agnostic. For now, the object writes themselves are still "files" backend specific due to being handled by the git-index-pack(1) and git-unpack-objects(1) child processes. This will be tackled in a separate series though. Changes since V2: - Clarified commit log reasoning for embedding `flush_loose_object_transaction()` logic in commit function. - Started printed some error messages on transaction errors. - Removed include statement. - Fixed transaction leak on `odb_transaction_commit()` error. Changes since V1: - Adapted other "file" ODB transaction helpers to be more consistent with current naming scheme. - Removed redundant NULL transaction handling from `odb_transaction_files_begin()`. - `odb_transaction_begin()` now returns an error if there is already an inflight transaction pending instead of setting the `out` pointer to NULL. - Updated `odb_transaction_env()` to return an error code and append environment variables to a strvec provided as an argument. - Removed redundant setting of tmpdir environment variables for child processes after tmpdir has been migrated. - Split changes adding ODB transaction flags into a separate commit. - Consistently wire the ODB transaction throughout git-receive-pack code instead of reading it from `the_repository`. - Updated user facing error message. - Updated some comments to better document functions/flags. - Clarified some commit messages. - Fixed typos. Thanks, -Justin Justin Tobler (11): object-file: rename files transaction prepare function object-file: rename files transaction fsync function object-file: embed transaction flush logic in commit function object-file: drop check for inflight transactions object-file: propagate files transaction errors odb/transaction: propagate begin errors odb/transaction: propagate commit errors odb/transaction: add transaction env interface odb/transaction: introduce ODB transaction flags builtin/receive-pack: drop redundant tmpdir env builtin/receive-pack: stage incoming objects via ODB transactions builtin/add.c | 2 +- builtin/receive-pack.c | 69 ++++++++--------- builtin/unpack-objects.c | 2 +- builtin/update-index.c | 2 +- cache-tree.c | 7 +- object-file.c | 159 +++++++++++++++++++++++++-------------- object-file.h | 8 +- odb/source-files.c | 9 +-- odb/source-inmemory.c | 3 +- odb/source-loose.c | 3 +- odb/source.h | 9 ++- odb/transaction.c | 32 ++++++-- odb/transaction.h | 59 ++++++++++++--- read-cache.c | 7 +- 14 files changed, 241 insertions(+), 130 deletions(-) Range-diff against v2: 1: 9c14b219ad = 1: 9c14b219ad object-file: rename files transaction prepare function 2: 5703a9e93b = 2: 5703a9e93b object-file: rename files transaction fsync function 3: 4c37398ac8 ! 3: 76204847f2 object-file: embed transaction flush logic in commit function @@ Commit message When a "files" transaction is committed, `flush_loose_object_transaction()` is invoked to handle performing a hardware flush along with migrating the temporary object directory into - the primary. In a subsequent commit, the temporary directory is also - used to write packfiles. + the primary and configuring the repository ODB source accordingly. The + function name here is a bit misleading because the helper is doing a bit + more than just "flushing" the transaction contents. Also, in a + subsequent commit, the transaction temporary directory is used to stage + packfiles and not just loose objects anymore. - Instead of maintaining a separate helper function, embed the logic to - flush and migrate the temporary directory directly into - `odb_transaction_files_commit()`. + Lift the helper function logic directly into + `odb_transaction_files_commit()` to more accurately signal to readers + the operation being performed. Signed-off-by: Justin Tobler 4: 623c6b02ea = 4: c97eb7763f object-file: drop check for inflight transactions 5: ca59176657 ! 5: 1f3a1f7714 object-file: propagate files transaction errors @@ object-file.c: static void odb_transaction_files_prepare(struct odb_transaction - if (transaction->objdir) - tmp_objdir_replace_primary_odb(transaction->objdir, 0); + if (!transaction->objdir) -+ return -1; ++ return error(_("unable to create temporary object directory")); + + tmp_objdir_replace_primary_odb(transaction->objdir, 0); + @@ object-file.c: static void odb_transaction_files_commit(struct odb_transaction * */ - tmp_objdir_migrate(transaction->objdir); + if (tmp_objdir_migrate(transaction->objdir)) -+ return -1; ++ return error(_("unable to migrate temporary objects")); + transaction->objdir = NULL; } 6: 717a1ce9a7 ! 6: 09d13272d5 odb/transaction: propagate begin errors @@ object-file.c: int index_fd(struct index_state *istate, struct object_id *oid, ## odb/transaction.c ## @@ + #include "git-compat-util.h" ++#include "gettext.h" #include "odb/source.h" #include "odb/transaction.h" @@ odb/transaction.c + if (odb->transaction) - return NULL; -+ return -1; ++ return error(_("object database transaction already pending")); - odb_source_begin_transaction(odb->sources, &odb->transaction); + ret = odb_source_begin_transaction(odb->sources, out); @@ odb/transaction.h #ifndef ODB_TRANSACTION_H #define ODB_TRANSACTION_H -+#include "git-compat-util.h" +#include "gettext.h" #include "odb.h" #include "odb/source.h" 7: ff8e133965 ! 7: 12833d6773 odb/transaction: propagate commit errors @@ odb/transaction.c: int odb_transaction_begin(struct object_database *odb, - transaction->commit(transaction); + ret = transaction->commit(transaction); -+ if (ret) -+ return ret; -+ transaction->source->odb->transaction = NULL; free(transaction); + -+ return 0; ++ return ret; } int odb_transaction_write_object_stream(struct odb_transaction *transaction, 8: 264ba94b83 = 8: f2586f2f34 odb/transaction: add transaction env interface 9: 1e0a491ef2 ! 9: 9d082b5e47 odb/transaction: introduce ODB transaction flags @@ object-file.c: static int odb_transaction_files_prepare(struct odb_transaction * - transaction->objdir = tmp_objdir_create(base->source->odb->repo, "bulk-fsync"); + transaction->objdir = tmp_objdir_create(base->source->odb->repo, transaction->prefix); if (!transaction->objdir) - return -1; + return error(_("unable to create temporary object directory")); @@ object-file.c: int index_fd(struct index_state *istate, struct object_id *oid, int inflight = !!transaction; @@ odb/transaction.c int ret; if (odb->transaction) - return -1; + return error(_("object database transaction already pending")); - ret = odb_source_begin_transaction(odb->sources, out); + ret = odb_source_begin_transaction(odb->sources, out, flags); @@ odb/transaction.c ## odb/transaction.h ## @@ - #include "git-compat-util.h" + #include "gettext.h" #include "odb.h" -#include "odb/source.h" 10: 6c8d878349 = 10: e11d8a6676 builtin/receive-pack: drop redundant tmpdir env 11: 8db95fef56 = 11: fee57c2817 builtin/receive-pack: stage incoming objects via ODB transactions base-commit: ab776a62a78576513ee121424adb19597fbb7613 -- 2.55.0.122.gf85a7e6620