From: Justin Tobler Date: Thu, 14 May 2026 18:37:33 GMT Subject: [PATCH v4 0/7] odb: add write operation to ODB transaction interface Message-ID: <20260514183740.1505171-1-jltobler@gmail.com> In-Reply-To: <20260402213220.2651523-1-jltobler@gmail.com> Greetings, This series lays the groundwork for introducing write operations to the ODB transaction interface. The eventual goal is for all object writes performed within a transaction to go through this interface explicitly, rather than implicitly relying on the transaction to reconfigure ODB sources so that writes are redirected to a temporary location. For now, only `odb_transaction_write_object_stream()` is implemented and wires up the existing logic for streaming "large" blobs directly into a packfile as part of the transaction. Most of the patches are structural refactorings to enable this, but patch 4 introduces a behavioral change in how packfiles that would exceed "pack.packSizeLimit" are handled. Changes since V3: - Fixed leak due to an fd not being closed when exiting prior to close_loose_object() being invoked. Changes since V2: - Renamed some variables to improve clarity - Make `odb_write_stream_from_fd()` fully initialize the underlying `struct odb_write_stream` - Move `struct odb_write_stream` to "odb/streaming.h" - Make the `hash_blob_stream()` helper more generic by operating on a `struct odb_write_stream` instead of reading from an fd directly. - Introduce an `odb_write_stream_release()` helper to free the underlying stream data. Changes since V1: - Fixed some typos - Improved error handling - Removed unnecessary guard statement - Documented in comments why inflated object size is used to approximate if object write will exceed "pack.packSizeLimit". - Updated `struct odb_write_stream` read() callback to support returning errors and using caller provided buffer - Updated the `hash_blob_stream()` function signature to operate on a `struct odb_write_stream` instead of an fd directly - Renamed some variables/functions for better clarity Thanks, -Justin Justin Tobler (7): odb: split `struct odb_transaction` into separate header odb/transaction: use pluggable `begin_transaction()` odb: update `struct odb_write_stream` read() callback object-file: remove flags from transaction packfile writes object-file: avoid fd seekback by checking object size upfront object-file: generalize packfile writes to use odb_write_stream odb/transaction: make `write_object_stream()` pluggable Makefile | 1 + builtin/add.c | 1 + builtin/unpack-objects.c | 21 ++-- builtin/update-index.c | 1 + cache-tree.c | 1 + meson.build | 1 + object-file.c | 238 ++++++++++++++++++++------------------- odb.c | 25 ---- odb.h | 37 +----- odb/streaming.c | 51 +++++++++ odb/streaming.h | 30 +++++ odb/transaction.c | 35 ++++++ odb/transaction.h | 57 ++++++++++ read-cache.c | 1 + 14 files changed, 312 insertions(+), 188 deletions(-) create mode 100644 odb/transaction.c create mode 100644 odb/transaction.h Range-diff against v3: 1: eee372b426 = 1: eee372b426 odb: split `struct odb_transaction` into separate header 2: 57ac075560 = 2: 57ac075560 odb/transaction: use pluggable `begin_transaction()` 3: 11321ad607 ! 3: d53ad95712 odb: update `struct odb_write_stream` read() callback @@ object-file.c: int odb_source_loose_write_stream(struct odb_source *source, + ssize_t read_len = odb_write_stream_read(in_stream, buf, + sizeof(buf)); + if (read_len < 0) { ++ close(fd); + err = -1; + goto cleanup; + } 4: 72d4656eee = 4: fa7a3ad5dc object-file: remove flags from transaction packfile writes 5: e4896101ff = 5: 1ca08e0590 object-file: avoid fd seekback by checking object size upfront 6: b3cb0a707c = 6: a548401057 object-file: generalize packfile writes to use odb_write_stream 7: e1d292a7ed = 7: 4765b1024a odb/transaction: make `write_object_stream()` pluggable base-commit: 5361983c075154725be47b65cca9a2421789e410 -- 2.54.0.105.g59ff4886a5