Re: [PATCH 0/6] refs: introduce support for partial reference transactions

From: Phillip Wood <phillip.wood123@gmail.com>
Date: Feb 11, 2025, 17:03 UTC
Message-ID: <4beb0359-763d-425d-b416-ac40bda59e2e@gmail.com>
In-Reply-To: <20250207-245-partially-atomic-ref-updates-v1-0-e6a3690ff23a@gmail.com>

Hi Karthik

On 07/02/2025 07:34, Karthik Nayak wrote:

> Git's reference updates are traditionally atomic

I'm nitpicking but the updates aren't actually atomic, if a transaction updates two refs then it is possible for another process to see the one ref pointing to the new value and the other pointing to the old value.

> - when updating
> multiple references in a transaction, either all updates succeed or none
> do. While this behavior is generally desirable,

Isn't that the whole point of transactions?

Show 7 quoted lines

> it can be limiting in> certain scenarios, particularly with the reftable backend where batching
> multiple reference updates is more efficient than performing them
> sequentially.
>  
> This series introduces support for partial reference transactions,
> allowing individual reference updates to fail while letting others
> proceed.

This sounds like it's abusing ref transactions to implement a performance optimization. I wonder if it would be better to provide that via a different interface than shares the same underling implementation as transactions. That would make it clear to someone reading the code that individual ref updates can fail without affecting the rest. Burying that detail in a flag makes it rather easy to miss.

Best Wishes

Phillip

Show 66 quoted lines

> This capability is exposed through git-update-ref's
> `--allow-partial` flag, which can be used in `--stdin` mode to batch
> updates and handle failures gracefully.
> The changes are structured to carefully build up this functionality:
> 
> First, we clean up and consolidate the reference update checking logic.
> This includes removing duplicate checks in the files backend and moving
> refname tracking to the generic layer, which simplifies the codebase and
> prepares it for the new feature.
> 
> We then restructure the reftable backend's transaction preparation code,
> extracting the update validation logic into a dedicated function. This
> not only improves code organization but sets the stage for implementing
> partial transaction support.
> 
> With this groundwork in place, we implement the core partial transaction
> support in the refs subsystem. This adds the necessary infrastructure to
> track and report rejected updates while allowing transactions to proceed.
> All reference backends are modified to support this behavior when enabled.
> 
> Finally, we expose this functionality to users through
> git-update-ref(1)'s `--allow-partial` flag, complete with test coverage
> and documentation. The flag is specifically limited to `--stdin` mode
> where batching multiple updates is most relevant.
> 
> This enhancement improves Git's flexibility in handling reference
> updates while maintaining the safety of atomic transactions by default.
> It's particularly valuable for tools and workflows that need to handle
> reference update failures gracefully without abandoning the entire batch
> of updates.
> 
> This series is based on top of bc204b7427 (The seventh batch, 2025-02-03).
> There were no conflicts noticed with topics in 'seen' or 'next'.
> 
> ---
> Karthik Nayak (6):
>        refs/files: remove duplicate check in `split_symref_update()`
>        refs: move duplicate refname update check to generic layer
>        refs/files: remove duplicate duplicates check
>        refs/reftable: extract code from the transaction preparation
>        refs: implement partial reference transaction support
>        update-ref: add --allow-partial flag for stdin mode
> 
>   Documentation/git-update-ref.txt |  12 +-
>   builtin/update-ref.c             |  53 ++++-
>   refs.c                           |  58 ++++-
>   refs.h                           |  22 ++
>   refs/files-backend.c             |  97 ++------
>   refs/packed-backend.c            |  49 ++--
>   refs/refs-internal.h             |  19 +-
>   refs/reftable-backend.c          | 494 +++++++++++++++++++--------------------
>   t/t1400-update-ref.sh            | 191 +++++++++++++++
>   9 files changed, 633 insertions(+), 362 deletions(-)
> ---
> 
> 
> 
> ---
> 
> base-commit: bc204b742735ae06f65bb20291c95985c9633b7f
> change-id: 20241206-245-partially-atomic-ref-updates-9fe8b080345c
> 
> Thanks
> - Karthik
> 
>