{"thread":{"id":"62915","subject":"[PATCH 0/6] refs: introduce support for partial reference transactions","startedAt":"2025-02-07T07:37:02Z","lastAt":"2025-04-10T11:23:02Z","messageCount":49,"participants":["Karthik Nayak","Patrick Steinhardt","Phillip Wood","Jean-Noël AVILA","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":6},"messages":[{"id":"512038","messageId":"20250207-245-partially-atomic-ref-updates-v1-0-e6a3690ff23a@gmail.com","threadId":"62915","inReplyTo":null,"subject":"[PATCH 0/6] refs: introduce support for partial reference transactions","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-02-07T07:34:35Z","receivedAt":"2025-02-07T07:37:02Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Git's reference updates are traditionally atomic - when updating\nmultiple references in a transaction, either all updates succeed or none\ndo. While this behavior is generally desirable, it can be limiting in\ncertain scenarios, particularly with the reftable backend where batching\nmultiple reference updates is more efficient than performing them\nsequentially.\n\nThis series introduces support for partial reference transactions,\nallowing individual reference updates to fail while letting others\nproceed. This capability is exposed through git-update-ref's\n`--allow-partial` flag, which can be used in `--stdin` mode to batch\nupdates and handle failures gracefully.\n\nThe changes are structured to carefully build up this functionality:\n\nFirst, we clean up and consolidate the reference update checking logic.\nThis includes removing duplicate checks in the files backend and moving\nrefname tracking to the generic layer, which simplifies the codebase and\nprepares it for the new feature.\n\nWe then restructure the reftable backend's transaction preparation code,\nextracting the update validation logic into a dedicated function. This\nnot only improves code organization but sets the stage for implementing\npartial transaction support.\n\nWith this groundwork in place, we implement the core partial transaction\nsupport in the refs subsystem. This adds the necessary infrastructure to\ntrack and report rejected updates while allowing transactions to proceed.\nAll reference backends are modified to support this behavior when enabled.\n\nFinally, we expose this functionality to users through\ngit-update-ref(1)'s `--allow-partial` flag, complete with test coverage\nand documentation. The flag is specifically limited to `--stdin` mode\nwhere batching multiple updates is most relevant.\n\nThis enhancement improves Git's flexibility in handling reference\nupdates while maintaining the safety of atomic transactions by default.\nIt's particularly valuable for tools and workflows that need to handle\nreference update failures gracefully without abandoning the entire batch\nof updates.\n\nThis series is based on top of bc204b7427 (The seventh batch, 2025-02-03).\nThere were no conflicts noticed with topics in 'seen' or 'next'.\n\n---\nKarthik Nayak (6):\n refs/files: remove duplicate check in `split_symref_update()`\n refs: move duplicate refname update check to generic layer\n refs/files: remove duplicate duplicates check\n refs/reftable: extract code from the transaction preparation\n refs: implement partial reference transaction support\n update-ref: add --allow-partial flag for stdin mode\n\n Documentation/git-update-ref.txt | 12 +-\n builtin/update-ref.c | 53 ++++-\n refs.c | 58 ++++-\n refs.h | 22 ++\n refs/files-backend.c | 97 ++------\n refs/packed-backend.c | 49 ++--\n refs/refs-internal.h | 19 +-\n refs/reftable-backend.c | 494 +++++++++++++++++++--------------------\n t/t1400-update-ref.sh | 191 +++++++++++++++\n 9 files changed, 633 insertions(+), 362 deletions(-)\n---\n\n\n\n---\n\nbase-commit: bc204b742735ae06f65bb20291c95985c9633b7f\nchange-id: 20241206-245-partially-atomic-ref-updates-9fe8b080345c\n\nThanks\n- Karthik\n\n"},{"id":"512039","messageId":"20250207-245-partially-atomic-ref-updates-v1-1-e6a3690ff23a@gmail.com","threadId":"62915","inReplyTo":"20250207-245-partially-atomic-ref-updates-v1-0-e6a3690ff23a@gmail.com","subject":"[PATCH 1/6] refs/files: remove duplicate check in `split_symref_update()`","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-02-07T07:34:36Z","receivedAt":"2025-02-07T07:37:04Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"In split_symref_update(), there were two redundant checks:\n - At the start: checking if refname exists in `affected_refnames`.\n - After adding refname: checking if the item added to\n `affected_refnames` contains the util field.\n\nRemove the second check since the first one already prevents duplicate\nrefnames from being added to the transaction updates.\n\nSince this is the only place that utilizes the `item->util` value, avoid\nsetting the value in the first place and cleanup code around it.\n\nSigned-off-by: Karthik Nayak \n---\n refs/files-backend.c | 20 +++-----------------\n 1 file changed, 3 insertions(+), 17 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 29f08dced40418eb815072c6335e0c3d1a45c7d8..c6a3f6d6261a894e1c294bb1329fdf8079a39eb4 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -2387,7 +2387,6 @@ static int split_head_update(struct ref_update *update,\n \t\t\t struct string_list *affected_refnames,\n \t\t\t struct strbuf *err)\n {\n-\tstruct string_list_item *item;\n \tstruct ref_update *new_update;\n \n \tif ((update->flags & REF_LOG_ONLY) ||\n@@ -2426,8 +2425,7 @@ static int split_head_update(struct ref_update *update,\n \t */\n \tif (strcmp(new_update->refname, \"HEAD\"))\n \t\tBUG(\"%s unexpectedly not 'HEAD'\", new_update->refname);\n-\titem = string_list_insert(affected_refnames, new_update->refname);\n-\titem->util = new_update;\n+\tstring_list_insert(affected_refnames, new_update->refname);\n \n \treturn 0;\n }\n@@ -2446,7 +2444,6 @@ static int split_symref_update(struct ref_update *update,\n \t\t\t struct string_list *affected_refnames,\n \t\t\t struct strbuf *err)\n {\n-\tstruct string_list_item *item;\n \tstruct ref_update *new_update;\n \tunsigned int new_flags;\n \n@@ -2501,11 +2498,7 @@ static int split_symref_update(struct ref_update *update,\n \t * be valid as long as affected_refnames is in use, and NOT\n \t * referent, which might soon be freed by our caller.\n \t */\n-\titem = string_list_insert(affected_refnames, new_update->refname);\n-\tif (item->util)\n-\t\tBUG(\"%s unexpectedly found in affected_refnames\",\n-\t\t new_update->refname);\n-\titem->util = new_update;\n+\tstring_list_insert(affected_refnames, new_update->refname);\n \n \treturn 0;\n }\n@@ -2837,7 +2830,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t */\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n-\t\tstruct string_list_item *item;\n \n \t\tif ((update->flags & REF_IS_PRUNING) &&\n \t\t !(update->flags & REF_NO_DEREF))\n@@ -2846,13 +2838,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t\tif (update->flags & REF_LOG_ONLY)\n \t\t\tcontinue;\n \n-\t\titem = string_list_append(&affected_refnames, update->refname);\n-\t\t/*\n-\t\t * We store a pointer to update in item->util, but at\n-\t\t * the moment we never use the value of this field\n-\t\t * except to check whether it is non-NULL.\n-\t\t */\n-\t\titem->util = update;\n+\t\tstring_list_append(&affected_refnames, update->refname);\n \t}\n \tstring_list_sort(&affected_refnames);\n \tif (ref_update_reject_duplicates(&affected_refnames, err)) {\n\n-- \n2.47.0\n\n"},{"id":"512040","messageId":"20250207-245-partially-atomic-ref-updates-v1-2-e6a3690ff23a@gmail.com","threadId":"62915","inReplyTo":"20250207-245-partially-atomic-ref-updates-v1-0-e6a3690ff23a@gmail.com","subject":"[PATCH 2/6] refs: move duplicate refname update check to generic layer","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-02-07T07:34:37Z","receivedAt":"2025-02-07T07:37:07Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Move the tracking of refnames in `affected_refnames` from individual\nbackends into the generic layer in 'refs.c'. This centralizes the\nduplicate refname detection that was previously handled separately by\neach backend.\n\nMake some changes to accommodate this move:\n\n - Add a `string_list` field `refnames` to `ref_transaction` to contain\n all the references in a transaction. This field is updated whenever\n a new update is added.\n\n - Modify the backends to use this field internally as needed. The\n backends need to check if an update for refname already exists when\n splitting symrefs or adding an update for 'HEAD'.\n\n - In the reftable backend, in `reftable_be_transaction_prepare()`,\n move the instance of `string_list_has_string()` above\n `ref_transaction_add_update()` to check before the reference is\n added.\n\nThis helps reduce duplication of functionality between the backends and\nmakes it easier to make changes in a more centralized manner.\n\nSigned-off-by: Karthik Nayak \n---\n refs.c | 17 ++++++++++++\n refs/files-backend.c | 69 ++++++++++---------------------------------------\n refs/packed-backend.c | 25 +-----------------\n refs/refs-internal.h | 2 ++\n refs/reftable-backend.c | 53 ++++++++++++-------------------------\n 5 files changed, 50 insertions(+), 116 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex f4094a326a9f88f979654b668cc9c3d27d83cb5d..4c9b706461977995be1d55e7667f7fb708fbbb76 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1175,6 +1175,7 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,\n \tCALLOC_ARRAY(tr, 1);\n \ttr->ref_store = refs;\n \ttr->flags = flags;\n+\tstring_list_init_dup(&tr->refnames);\n \treturn tr;\n }\n \n@@ -1205,6 +1206,7 @@ void ref_transaction_free(struct ref_transaction *transaction)\n \t\tfree((char *)transaction->updates[i]->old_target);\n \t\tfree(transaction->updates[i]);\n \t}\n+\tstring_list_clear(&transaction->refnames, 0);\n \tfree(transaction->updates);\n \tfree(transaction);\n }\n@@ -1218,6 +1220,7 @@ struct ref_update *ref_transaction_add_update(\n \t\tconst char *committer_info,\n \t\tconst char *msg)\n {\n+\tstruct string_list_item *item;\n \tstruct ref_update *update;\n \n \tif (transaction->state != REF_TRANSACTION_OPEN)\n@@ -1245,6 +1248,16 @@ struct ref_update *ref_transaction_add_update(\n \t\tupdate->msg = normalize_reflog_message(msg);\n \t}\n \n+\t/*\n+\t * This list is generally used by the backends to avoid duplicates.\n+\t * But we do support multiple log updates for a given refname within\n+\t * a single transaction.\n+\t */\n+\tif (!(update->flags & REF_LOG_ONLY)) {\n+\t\titem = string_list_append(&transaction->refnames, refname);\n+\t\titem->util = update;\n+\t}\n+\n \treturn update;\n }\n \n@@ -2397,6 +2410,10 @@ int ref_transaction_prepare(struct ref_transaction *transaction,\n \t\treturn -1;\n \t}\n \n+\tstring_list_sort(&transaction->refnames);\n+\tif (ref_update_reject_duplicates(&transaction->refnames, err))\n+\t\treturn TRANSACTION_GENERIC_ERROR;\n+\n \tret = refs->be->transaction_prepare(refs, transaction, err);\n \tif (ret)\n \t\treturn ret;\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex c6a3f6d6261a894e1c294bb1329fdf8079a39eb4..18da30c3f37dc5c09f7d81a9083d6b41d0463bd5 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -2383,9 +2383,7 @@ static struct ref_iterator *files_reflog_iterator_begin(struct ref_store *ref_st\n */\n static int split_head_update(struct ref_update *update,\n \t\t\t struct ref_transaction *transaction,\n-\t\t\t const char *head_ref,\n-\t\t\t struct string_list *affected_refnames,\n-\t\t\t struct strbuf *err)\n+\t\t\t const char *head_ref, struct strbuf *err)\n {\n \tstruct ref_update *new_update;\n \n@@ -2403,7 +2401,7 @@ static int split_head_update(struct ref_update *update,\n \t * transaction. This check is O(lg N) in the transaction\n \t * size, but it happens at most once per transaction.\n \t */\n-\tif (string_list_has_string(affected_refnames, \"HEAD\")) {\n+\tif (string_list_has_string(&transaction->refnames, \"HEAD\")) {\n \t\t/* An entry already existed */\n \t\tstrbuf_addf(err,\n \t\t\t \"multiple updates for 'HEAD' (including one \"\n@@ -2425,7 +2423,6 @@ static int split_head_update(struct ref_update *update,\n \t */\n \tif (strcmp(new_update->refname, \"HEAD\"))\n \t\tBUG(\"%s unexpectedly not 'HEAD'\", new_update->refname);\n-\tstring_list_insert(affected_refnames, new_update->refname);\n \n \treturn 0;\n }\n@@ -2441,7 +2438,6 @@ static int split_head_update(struct ref_update *update,\n static int split_symref_update(struct ref_update *update,\n \t\t\t const char *referent,\n \t\t\t struct ref_transaction *transaction,\n-\t\t\t struct string_list *affected_refnames,\n \t\t\t struct strbuf *err)\n {\n \tstruct ref_update *new_update;\n@@ -2453,7 +2449,7 @@ static int split_symref_update(struct ref_update *update,\n \t * size, but it happens at most once per symref in a\n \t * transaction.\n \t */\n-\tif (string_list_has_string(affected_refnames, referent)) {\n+\tif (string_list_has_string(&transaction->refnames, referent)) {\n \t\t/* An entry already exists */\n \t\tstrbuf_addf(err,\n \t\t\t \"multiple updates for '%s' (including one \"\n@@ -2491,15 +2487,6 @@ static int split_symref_update(struct ref_update *update,\n \tupdate->flags |= REF_LOG_ONLY | REF_NO_DEREF;\n \tupdate->flags &= ~REF_HAVE_OLD;\n \n-\t/*\n-\t * Add the referent. This insertion is O(N) in the transaction\n-\t * size, but it happens at most once per symref in a\n-\t * transaction. Make sure to add new_update->refname, which will\n-\t * be valid as long as affected_refnames is in use, and NOT\n-\t * referent, which might soon be freed by our caller.\n-\t */\n-\tstring_list_insert(affected_refnames, new_update->refname);\n-\n \treturn 0;\n }\n \n@@ -2561,9 +2548,7 @@ struct files_transaction_backend_data {\n static int lock_ref_for_update(struct files_ref_store *refs,\n \t\t\t struct ref_update *update,\n \t\t\t struct ref_transaction *transaction,\n-\t\t\t const char *head_ref,\n-\t\t\t struct string_list *affected_refnames,\n-\t\t\t struct strbuf *err)\n+\t\t\t const char *head_ref, struct strbuf *err)\n {\n \tstruct strbuf referent = STRBUF_INIT;\n \tint mustexist = ref_update_expects_existing_old_ref(update);\n@@ -2579,8 +2564,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\tupdate->flags |= REF_DELETING;\n \n \tif (head_ref) {\n-\t\tret = split_head_update(update, transaction, head_ref,\n-\t\t\t\t\taffected_refnames, err);\n+\t\tret = split_head_update(update, transaction, head_ref, err);\n \t\tif (ret)\n \t\t\tgoto out;\n \t}\n@@ -2590,7 +2574,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\tlock->count++;\n \t} else {\n \t\tret = lock_raw_ref(refs, update->refname, mustexist,\n-\t\t\t\t affected_refnames,\n+\t\t\t\t &transaction->refnames,\n \t\t\t\t &lock, &referent,\n \t\t\t\t &update->type, err);\n \t\tif (ret) {\n@@ -2646,9 +2630,8 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\t\t * of processing the split-off update, so we\n \t\t\t * don't have to do it here.\n \t\t\t */\n-\t\t\tret = split_symref_update(update,\n-\t\t\t\t\t\t referent.buf, transaction,\n-\t\t\t\t\t\t affected_refnames, err);\n+\t\t\tret = split_symref_update(update, referent.buf,\n+\t\t\t\t\t\t transaction, err);\n \t\t\tif (ret)\n \t\t\t\tgoto out;\n \t\t}\n@@ -2803,7 +2786,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t\t\t \"ref_transaction_prepare\");\n \tsize_t i;\n \tint ret = 0;\n-\tstruct string_list affected_refnames = STRING_LIST_INIT_NODUP;\n \tchar *head_ref = NULL;\n \tint head_type;\n \tstruct files_transaction_backend_data *backend_data;\n@@ -2821,12 +2803,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \ttransaction->backend_data = backend_data;\n \n \t/*\n-\t * Fail if a refname appears more than once in the\n-\t * transaction. (If we end up splitting up any updates using\n-\t * split_symref_update() or split_head_update(), those\n-\t * functions will check that the new updates don't have the\n-\t * same refname as any existing ones.) Also fail if any of the\n-\t * updates use REF_IS_PRUNING without REF_NO_DEREF.\n+\t * Fail if any of the updates use REF_IS_PRUNING without REF_NO_DEREF.\n \t */\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n@@ -2834,16 +2811,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t\tif ((update->flags & REF_IS_PRUNING) &&\n \t\t !(update->flags & REF_NO_DEREF))\n \t\t\tBUG(\"REF_IS_PRUNING set without REF_NO_DEREF\");\n-\n-\t\tif (update->flags & REF_LOG_ONLY)\n-\t\t\tcontinue;\n-\n-\t\tstring_list_append(&affected_refnames, update->refname);\n-\t}\n-\tstring_list_sort(&affected_refnames);\n-\tif (ref_update_reject_duplicates(&affected_refnames, err)) {\n-\t\tret = TRANSACTION_GENERIC_ERROR;\n-\t\tgoto cleanup;\n \t}\n \n \t/*\n@@ -2884,7 +2851,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t\tstruct ref_update *update = transaction->updates[i];\n \n \t\tret = lock_ref_for_update(refs, update, transaction,\n-\t\t\t\t\t head_ref, &affected_refnames, err);\n+\t\t\t\t\t head_ref, err);\n \t\tif (ret)\n \t\t\tgoto cleanup;\n \n@@ -2957,7 +2924,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \n cleanup:\n \tfree(head_ref);\n-\tstring_list_clear(&affected_refnames, 0);\n \n \tif (ret)\n \t\tfiles_transaction_cleanup(refs, transaction);\n@@ -3021,7 +2987,6 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n {\n \tsize_t i;\n \tint ret = 0;\n-\tstruct string_list affected_refnames = STRING_LIST_INIT_NODUP;\n \tstruct ref_transaction *packed_transaction = NULL;\n \tstruct ref_transaction *loose_transaction = NULL;\n \n@@ -3030,13 +2995,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \tif (transaction->state != REF_TRANSACTION_PREPARED)\n \t\tBUG(\"commit called for transaction that is not prepared\");\n \n-\t/* Fail if a refname appears more than once in the transaction: */\n-\tfor (i = 0; i < transaction->nr; i++)\n-\t\tif (!(transaction->updates[i]->flags & REF_LOG_ONLY))\n-\t\t\tstring_list_append(&affected_refnames,\n-\t\t\t\t\t transaction->updates[i]->refname);\n-\tstring_list_sort(&affected_refnames);\n-\tif (ref_update_reject_duplicates(&affected_refnames, err)) {\n+\tstring_list_sort(&transaction->refnames);\n+\tif (ref_update_reject_duplicates(&transaction->refnames, err)) {\n \t\tret = TRANSACTION_GENERIC_ERROR;\n \t\tgoto cleanup;\n \t}\n@@ -3054,7 +3014,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \t * that we are creating already exists.\n \t */\n \tif (refs_for_each_rawref(&refs->base, ref_present,\n-\t\t\t\t &affected_refnames))\n+\t\t\t\t &transaction->refnames))\n \t\tBUG(\"initial ref transaction called with existing refs\");\n \n \tpacked_transaction = ref_store_transaction_begin(refs->packed_ref_store,\n@@ -3072,7 +3032,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \t\t\tBUG(\"initial ref transaction with old_sha1 set\");\n \n \t\tif (refs_verify_refname_available(&refs->base, update->refname,\n-\t\t\t\t\t\t &affected_refnames, NULL, 1, err)) {\n+\t\t\t\t\t\t &transaction->refnames, NULL, 1, err)) {\n \t\t\tret = TRANSACTION_NAME_CONFLICT;\n \t\t\tgoto cleanup;\n \t\t}\n@@ -3132,7 +3092,6 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \tif (packed_transaction)\n \t\tref_transaction_free(packed_transaction);\n \ttransaction->state = REF_TRANSACTION_CLOSED;\n-\tstring_list_clear(&affected_refnames, 0);\n \treturn ret;\n }\n \ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex a7b6f74b6e35f897f619c540cbc600bbd888bc67..6e7acb077e81435715a1ca3cc928550147c8c56a 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1604,8 +1604,6 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n struct packed_transaction_backend_data {\n \t/* True iff the transaction owns the packed-refs lock. */\n \tint own_lock;\n-\n-\tstruct string_list updates;\n };\n \n static void packed_transaction_cleanup(struct packed_ref_store *refs,\n@@ -1614,8 +1612,6 @@ static void packed_transaction_cleanup(struct packed_ref_store *refs,\n \tstruct packed_transaction_backend_data *data = transaction->backend_data;\n \n \tif (data) {\n-\t\tstring_list_clear(&data->updates, 0);\n-\n \t\tif (is_tempfile_active(refs->tempfile))\n \t\t\tdelete_tempfile(&refs->tempfile);\n \n@@ -1640,7 +1636,6 @@ static int packed_transaction_prepare(struct ref_store *ref_store,\n \t\t\tREF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,\n \t\t\t\"ref_transaction_prepare\");\n \tstruct packed_transaction_backend_data *data;\n-\tsize_t i;\n \tint ret = TRANSACTION_GENERIC_ERROR;\n \n \t/*\n@@ -1653,34 +1648,16 @@ static int packed_transaction_prepare(struct ref_store *ref_store,\n \t */\n \n \tCALLOC_ARRAY(data, 1);\n-\tstring_list_init_nodup(&data->updates);\n \n \ttransaction->backend_data = data;\n \n-\t/*\n-\t * Stick the updates in a string list by refname so that we\n-\t * can sort them:\n-\t */\n-\tfor (i = 0; i < transaction->nr; i++) {\n-\t\tstruct ref_update *update = transaction->updates[i];\n-\t\tstruct string_list_item *item =\n-\t\t\tstring_list_append(&data->updates, update->refname);\n-\n-\t\t/* Store a pointer to update in item->util: */\n-\t\titem->util = update;\n-\t}\n-\tstring_list_sort(&data->updates);\n-\n-\tif (ref_update_reject_duplicates(&data->updates, err))\n-\t\tgoto failure;\n-\n \tif (!is_lock_file_locked(&refs->lock)) {\n \t\tif (packed_refs_lock(ref_store, 0, err))\n \t\t\tgoto failure;\n \t\tdata->own_lock = 1;\n \t}\n \n-\tif (write_with_updates(refs, &data->updates, err))\n+\tif (write_with_updates(refs, &transaction->refnames, err))\n \t\tgoto failure;\n \n \ttransaction->state = REF_TRANSACTION_PREPARED;\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex aaab711bb96844755dfa600d37efdb25b30a0765..f43766e1f00443eb689685cf4df0fa0b80018a03 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -3,6 +3,7 @@\n \n #include \"refs.h\"\n #include \"iterator.h\"\n+#include \"string-list.h\"\n \n struct fsck_options;\n struct ref_transaction;\n@@ -198,6 +199,7 @@ enum ref_transaction_state {\n struct ref_transaction {\n \tstruct ref_store *ref_store;\n \tstruct ref_update **updates;\n+\tstruct string_list refnames;\n \tsize_t alloc;\n \tsize_t nr;\n \tenum ref_transaction_state state;\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex d39a14c5a469d7d219362e9eae4f578784d65a5b..dd2099d94948a4f23fd9f7ddc06bf3d741229eba 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -1068,7 +1068,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \tstruct reftable_ref_store *refs =\n \t\treftable_be_downcast(ref_store, REF_STORE_WRITE|REF_STORE_MAIN, \"ref_transaction_prepare\");\n \tstruct strbuf referent = STRBUF_INIT, head_referent = STRBUF_INIT;\n-\tstruct string_list affected_refnames = STRING_LIST_INIT_NODUP;\n \tstruct reftable_transaction_data *tx_data = NULL;\n \tstruct reftable_backend *be;\n \tstruct object_id head_oid;\n@@ -1092,10 +1091,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\t\t\t transaction->updates[i], err);\n \t\tif (ret)\n \t\t\tgoto done;\n-\n-\t\tif (!(transaction->updates[i]->flags & REF_LOG_ONLY))\n-\t\t\tstring_list_append(&affected_refnames,\n-\t\t\t\t\t transaction->updates[i]->refname);\n \t}\n \n \t/*\n@@ -1107,17 +1102,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\ttx_data->args[i].updates_alloc = tx_data->args[i].updates_expected;\n \t}\n \n-\t/*\n-\t * Fail if a refname appears more than once in the transaction.\n-\t * This code is taken from the files backend and is a good candidate to\n-\t * be moved into the generic layer.\n-\t */\n-\tstring_list_sort(&affected_refnames);\n-\tif (ref_update_reject_duplicates(&affected_refnames, err)) {\n-\t\tret = TRANSACTION_GENERIC_ERROR;\n-\t\tgoto done;\n-\t}\n-\n \t/*\n \t * TODO: it's dubious whether we should reload the stack that \"HEAD\"\n \t * belongs to or not. In theory, it may happen that we only modify\n@@ -1185,14 +1169,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t !(u->flags & REF_LOG_ONLY) &&\n \t\t !(u->flags & REF_UPDATE_VIA_HEAD) &&\n \t\t !strcmp(rewritten_ref, head_referent.buf)) {\n-\t\t\tstruct ref_update *new_update;\n-\n \t\t\t/*\n \t\t\t * First make sure that HEAD is not already in the\n \t\t\t * transaction. This check is O(lg N) in the transaction\n \t\t\t * size, but it happens at most once per transaction.\n \t\t\t */\n-\t\t\tif (string_list_has_string(&affected_refnames, \"HEAD\")) {\n+\t\t\tif (string_list_has_string(&transaction->refnames, \"HEAD\")) {\n \t\t\t\t/* An entry already existed */\n \t\t\t\tstrbuf_addf(err,\n \t\t\t\t\t _(\"multiple updates for 'HEAD' (including one \"\n@@ -1202,12 +1184,11 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\tgoto done;\n \t\t\t}\n \n-\t\t\tnew_update = ref_transaction_add_update(\n-\t\t\t\t\ttransaction, \"HEAD\",\n-\t\t\t\t\tu->flags | REF_LOG_ONLY | REF_NO_DEREF,\n-\t\t\t\t\t&u->new_oid, &u->old_oid, NULL, NULL, NULL,\n-\t\t\t\t\tu->msg);\n-\t\t\tstring_list_insert(&affected_refnames, new_update->refname);\n+\t\t\tref_transaction_add_update(\n+\t\t\t\ttransaction, \"HEAD\",\n+\t\t\t\tu->flags | REF_LOG_ONLY | REF_NO_DEREF,\n+\t\t\t\t&u->new_oid, &u->old_oid, NULL, NULL, NULL,\n+\t\t\t\tu->msg);\n \t\t}\n \n \t\tret = reftable_backend_read_ref(be, rewritten_ref,\n@@ -1225,7 +1206,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t * at a later point.\n \t\t\t */\n \t\t\tret = refs_verify_refname_available(ref_store, u->refname,\n-\t\t\t\t\t\t\t &affected_refnames, NULL,\n+\t\t\t\t\t\t\t &transaction->refnames, NULL,\n \t\t\t\t\t\t\t transaction->flags & REF_TRANSACTION_FLAG_INITIAL,\n \t\t\t\t\t\t\t err);\n \t\t\tif (ret < 0)\n@@ -1277,6 +1258,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\tif (!strcmp(rewritten_ref, \"HEAD\"))\n \t\t\t\t\tnew_flags |= REF_UPDATE_VIA_HEAD;\n \n+\t\t\t\tif (string_list_has_string(&transaction->refnames, referent.buf)) {\n+\t\t\t\t\tstrbuf_addf(err,\n+\t\t\t\t\t\t _(\"multiple updates for '%s' (including one \"\n+\t\t\t\t\t\t \"via symref '%s') are not allowed\"),\n+\t\t\t\t\t\t referent.buf, u->refname);\n+\t\t\t\t\tret = TRANSACTION_NAME_CONFLICT;\n+\t\t\t\t\tgoto done;\n+\t\t\t\t}\n+\n \t\t\t\t/*\n \t\t\t\t * If we are updating a symref (eg. HEAD), we should also\n \t\t\t\t * update the branch that the symref points to.\n@@ -1301,16 +1291,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\t */\n \t\t\t\tu->flags |= REF_LOG_ONLY | REF_NO_DEREF;\n \t\t\t\tu->flags &= ~REF_HAVE_OLD;\n-\n-\t\t\t\tif (string_list_has_string(&affected_refnames, new_update->refname)) {\n-\t\t\t\t\tstrbuf_addf(err,\n-\t\t\t\t\t\t _(\"multiple updates for '%s' (including one \"\n-\t\t\t\t\t\t \"via symref '%s') are not allowed\"),\n-\t\t\t\t\t\t referent.buf, u->refname);\n-\t\t\t\t\tret = TRANSACTION_NAME_CONFLICT;\n-\t\t\t\t\tgoto done;\n-\t\t\t\t}\n-\t\t\t\tstring_list_insert(&affected_refnames, new_update->refname);\n \t\t\t}\n \t\t}\n \n@@ -1391,7 +1371,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\tstrbuf_addf(err, _(\"reftable: transaction prepare: %s\"),\n \t\t\t\t reftable_error_str(ret));\n \t}\n-\tstring_list_clear(&affected_refnames, 0);\n \tstrbuf_release(&referent);\n \tstrbuf_release(&head_referent);\n \n\n-- \n2.47.0\n\n"},{"id":"512041","messageId":"20250207-245-partially-atomic-ref-updates-v1-3-e6a3690ff23a@gmail.com","threadId":"62915","inReplyTo":"20250207-245-partially-atomic-ref-updates-v1-0-e6a3690ff23a@gmail.com","subject":"[PATCH 3/6] refs/files: remove duplicate duplicates check","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-02-07T07:34:38Z","receivedAt":"2025-02-07T07:37:10Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Within the files reference backend's transaction's 'finish' phase, a\nverification step is currently performed wherein the refnames list is\nsorted and examined for multiple updates targeting the same refname.\n\nIt has been observed that this verification is redundant, as an\nidentical check is already executed during the transaction's 'prepare'\nstage. Since the refnames list remains unmodified following the\n'prepare' stage, this secondary verification can be safely eliminated.\n\nThe duplicate check has been removed accordingly, and the\n`ref_update_reject_duplicates()` function has been marked as static, as\nits usage is now confined to 'refs.c'.\n\nSigned-off-by: Karthik Nayak \n---\n refs.c | 9 +++++++--\n refs/files-backend.c | 6 ------\n refs/refs-internal.h | 8 --------\n 3 files changed, 7 insertions(+), 16 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 4c9b706461977995be1d55e7667f7fb708fbbb76..b420a120102b3793168598b885bba68e4f5f5f03 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2295,8 +2295,13 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,\n \treturn ret;\n }\n \n-int ref_update_reject_duplicates(struct string_list *refnames,\n-\t\t\t\t struct strbuf *err)\n+/*\n+ * Write an error to `err` and return a nonzero value iff the same\n+ * refname appears multiple times in `refnames`. `refnames` must be\n+ * sorted on entry to this function.\n+ */\n+static int ref_update_reject_duplicates(struct string_list *refnames,\n+\t\t\t\t\tstruct strbuf *err)\n {\n \tsize_t i, n = refnames->nr;\n \ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 18da30c3f37dc5c09f7d81a9083d6b41d0463bd5..9fc5454678340dd7c72539bfa0f15ee7eb24b1ff 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -2995,12 +2995,6 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \tif (transaction->state != REF_TRANSACTION_PREPARED)\n \t\tBUG(\"commit called for transaction that is not prepared\");\n \n-\tstring_list_sort(&transaction->refnames);\n-\tif (ref_update_reject_duplicates(&transaction->refnames, err)) {\n-\t\tret = TRANSACTION_GENERIC_ERROR;\n-\t\tgoto cleanup;\n-\t}\n-\n \t/*\n \t * It's really undefined to call this function in an active\n \t * repository or when there are existing references: we are\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex f43766e1f00443eb689685cf4df0fa0b80018a03..434362b6099a35f92906a04ddd65365140147572 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -142,14 +142,6 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t struct object_id *oid, struct strbuf *referent,\n \t\t unsigned int *type, int *failure_errno);\n \n-/*\n- * Write an error to `err` and return a nonzero value iff the same\n- * refname appears multiple times in `refnames`. `refnames` must be\n- * sorted on entry to this function.\n- */\n-int ref_update_reject_duplicates(struct string_list *refnames,\n-\t\t\t\t struct strbuf *err);\n-\n /*\n * Add a ref_update with the specified properties to transaction, and\n * return a pointer to the new object. This function does not verify\n\n-- \n2.47.0\n\n"},{"id":"512042","messageId":"20250207-245-partially-atomic-ref-updates-v1-4-e6a3690ff23a@gmail.com","threadId":"62915","inReplyTo":"20250207-245-partially-atomic-ref-updates-v1-0-e6a3690ff23a@gmail.com","subject":"[PATCH 4/6] refs/reftable: extract code from the transaction preparation","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-02-07T07:34:39Z","receivedAt":"2025-02-07T07:37:12Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Extract the core logic for preparing individual reference updates from\n`reftable_be_transaction_prepare()` into `prepare_single_update()`. This\ndedicated function now handles all validation and preparation steps for\neach reference update in the transaction, including object ID\nverification, HEAD reference handling, and symref processing.\n\nThe refactoring consolidates all reference update validation into a\nsingle logical block, which improves code maintainability and\nreadability. More importantly, this restructuring lays the groundwork\nfor implementing partial transaction support in the reftable backend,\nwhich will be introduced in the following commit.\n\nNo functional changes are included in this commit - it is purely a code\nreorganization to support future enhancements.\n\nSigned-off-by: Karthik Nayak \n---\n refs/reftable-backend.c | 471 ++++++++++++++++++++++++------------------------\n 1 file changed, 240 insertions(+), 231 deletions(-)\n\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex dd2099d94948a4f23fd9f7ddc06bf3d741229eba..5533acfaf9027765d5a270abfce96225e42cc823 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -1061,6 +1061,242 @@ static int queue_transaction_update(struct reftable_ref_store *refs,\n \treturn 0;\n }\n \n+static int prepare_single_update(struct ref_store *ref_store,\n+\t\t\t\t struct reftable_ref_store *refs,\n+\t\t\t\t struct reftable_transaction_data *tx_data,\n+\t\t\t\t struct ref_transaction *transaction,\n+\t\t\t\t struct reftable_backend *be,\n+\t\t\t\t struct ref_update *u,\n+\t\t\t\t unsigned int head_type,\n+\t\t\t\t struct strbuf *head_referent,\n+\t\t\t\t struct strbuf *referent,\n+\t\t\t\t struct strbuf *err)\n+{\n+\tstruct object_id current_oid = {0};\n+\tconst char *rewritten_ref;\n+\tint ret = 0;\n+\n+\t/*\n+\t * There is no need to reload the respective backends here as\n+\t * we have already reloaded them when preparing the transaction\n+\t * update. And given that the stacks have been locked there\n+\t * shouldn't have been any concurrent modifications of the\n+\t * stack.\n+\t */\n+\tret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\t/* Verify that the new object ID is valid. */\n+\tif ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&\n+\t !(u->flags & REF_SKIP_OID_VERIFICATION) &&\n+\t !(u->flags & REF_LOG_ONLY)) {\n+\t\tstruct object *o = parse_object(refs->base.repo, &u->new_oid);\n+\t\tif (!o) {\n+\t\t\tstrbuf_addf(err,\n+\t\t\t\t _(\"trying to write ref '%s' with nonexistent object %s\"),\n+\t\t\t\t u->refname, oid_to_hex(&u->new_oid));\n+\t\t\treturn -1;\n+\t\t}\n+\n+\t\tif (o->type != OBJ_COMMIT && is_branch(u->refname)) {\n+\t\t\tstrbuf_addf(err, _(\"trying to write non-commit object %s to branch '%s'\"),\n+\t\t\t\t oid_to_hex(&u->new_oid), u->refname);\n+\t\t\treturn -1;\n+\t\t}\n+\t}\n+\n+\t/*\n+\t * When we update the reference that HEAD points to we enqueue\n+\t * a second log-only update for HEAD so that its reflog is\n+\t * updated accordingly.\n+\t */\n+\tif (head_type == REF_ISSYMREF &&\n+\t !(u->flags & REF_LOG_ONLY) &&\n+\t !(u->flags & REF_UPDATE_VIA_HEAD) &&\n+\t !strcmp(rewritten_ref, head_referent->buf)) {\n+\t\t/*\n+\t\t * First make sure that HEAD is not already in the\n+\t\t * transaction. This check is O(lg N) in the transaction\n+\t\t * size, but it happens at most once per transaction.\n+\t\t */\n+\t\tif (string_list_has_string(&transaction->refnames, \"HEAD\")) {\n+\t\t\t/* An entry already existed */\n+\t\t\tstrbuf_addf(err,\n+\t\t\t\t _(\"multiple updates for 'HEAD' (including one \"\n+\t\t\t\t \"via its referent '%s') are not allowed\"),\n+\t\t\t\t u->refname);\n+\t\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t\t}\n+\n+\t\tref_transaction_add_update(\n+\t\t\ttransaction, \"HEAD\",\n+\t\t\tu->flags | REF_LOG_ONLY | REF_NO_DEREF,\n+\t\t\t&u->new_oid, &u->old_oid, NULL, NULL, NULL,\n+\t\t\tu->msg);\n+\t}\n+\n+\tret = reftable_backend_read_ref(be, rewritten_ref,\n+\t\t\t\t\t¤t_oid, referent, &u->type);\n+\tif (ret < 0)\n+\t\treturn ret;\n+\tif (ret > 0 && !ref_update_expects_existing_old_ref(u)) {\n+\t\t/*\n+\t\t * The reference does not exist, and we either have no\n+\t\t * old object ID or expect the reference to not exist.\n+\t\t * We can thus skip below safety checks as well as the\n+\t\t * symref splitting. But we do want to verify that\n+\t\t * there is no conflicting reference here so that we\n+\t\t * can output a proper error message instead of failing\n+\t\t * at a later point.\n+\t\t */\n+\t\tret = refs_verify_refname_available(ref_store, u->refname,\n+\t\t\t\t\t\t &transaction->refnames, NULL,\n+\t\t\t\t\t\t transaction->flags & REF_TRANSACTION_FLAG_INITIAL,\n+\t\t\t\t\t\t err);\n+\t\tif (ret < 0)\n+\t\t\treturn ret;\n+\n+\t\t/*\n+\t\t * There is no need to write the reference deletion\n+\t\t * when the reference in question doesn't exist.\n+\t\t */\n+\t\tif ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {\n+\t\t\tret = queue_transaction_update(refs, tx_data, u,\n+\t\t\t\t\t\t ¤t_oid, err);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\t\t}\n+\n+\t\treturn 0;\n+\t}\n+\tif (ret > 0) {\n+\t\t/* The reference does not exist, but we expected it to. */\n+\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n+\t\t\t\t \"unable to resolve reference '%s'\"),\n+\t\t\t ref_update_original_update_refname(u), u->refname);\n+\t\treturn -1;\n+\t}\n+\n+\tif (u->type & REF_ISSYMREF) {\n+\t\t/*\n+\t\t * The reftable stack is locked at this point already,\n+\t\t * so it is safe to call `refs_resolve_ref_unsafe()`\n+\t\t * here without causing races.\n+\t\t */\n+\t\tconst char *resolved = refs_resolve_ref_unsafe(&refs->base, u->refname, 0,\n+\t\t\t\t\t\t\t ¤t_oid, NULL);\n+\n+\t\tif (u->flags & REF_NO_DEREF) {\n+\t\t\tif (u->flags & REF_HAVE_OLD && !resolved) {\n+\t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n+\t\t\t\t\t\t \"error reading reference\"), u->refname);\n+\t\t\t\treturn -1;\n+\t\t\t}\n+\t\t} else {\n+\t\t\tstruct ref_update *new_update;\n+\t\t\tint new_flags;\n+\n+\t\t\tnew_flags = u->flags;\n+\t\t\tif (!strcmp(rewritten_ref, \"HEAD\"))\n+\t\t\t\tnew_flags |= REF_UPDATE_VIA_HEAD;\n+\n+\t\t\tif (string_list_has_string(&transaction->refnames, referent->buf)) {\n+\t\t\t\tstrbuf_addf(err,\n+\t\t\t\t\t _(\"multiple updates for '%s' (including one \"\n+\t\t\t\t\t \"via symref '%s') are not allowed\"),\n+\t\t\t\t\t referent->buf, u->refname);\n+\t\t\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t\t\t}\n+\n+\t\t\t/*\n+\t\t\t * If we are updating a symref (eg. HEAD), we should also\n+\t\t\t * update the branch that the symref points to.\n+\t\t\t *\n+\t\t\t * This is generic functionality, and would be better\n+\t\t\t * done in refs.c, but the current implementation is\n+\t\t\t * intertwined with the locking in files-backend.c.\n+\t\t\t */\n+\t\t\tnew_update = ref_transaction_add_update(\n+\t\t\t\ttransaction, referent->buf, new_flags,\n+\t\t\t\tu->new_target ? NULL : &u->new_oid,\n+\t\t\t\tu->old_target ? NULL : &u->old_oid,\n+\t\t\t\tu->new_target, u->old_target,\n+\t\t\t\tu->committer_info, u->msg);\n+\n+\t\t\tnew_update->parent_update = u;\n+\n+\t\t\t/*\n+\t\t\t * Change the symbolic ref update to log only. Also, it\n+\t\t\t * doesn't need to check its old OID value, as that will be\n+\t\t\t * done when new_update is processed.\n+\t\t\t */\n+\t\t\tu->flags |= REF_LOG_ONLY | REF_NO_DEREF;\n+\t\t\tu->flags &= ~REF_HAVE_OLD;\n+\t\t}\n+\t}\n+\n+\t/*\n+\t * Verify that the old object matches our expectations. Note\n+\t * that the error messages here do not make a lot of sense in\n+\t * the context of the reftable backend as we never lock\n+\t * individual refs. But the error messages match what the files\n+\t * backend returns, which keeps our tests happy.\n+\t */\n+\tif (u->old_target) {\n+\t\tif (!(u->type & REF_ISSYMREF)) {\n+\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n+\t\t\t\t\t \"expected symref with target '%s': \"\n+\t\t\t\t\t \"but is a regular ref\"),\n+\t\t\t\t ref_update_original_update_refname(u),\n+\t\t\t\t u->old_target);\n+\t\t\treturn -1;\n+\t\t}\n+\n+\t\tif (ref_update_check_old_target(referent->buf, u, err)) {\n+\t\t\treturn -1;\n+\t\t}\n+\t} else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {\n+\t\tif (is_null_oid(&u->old_oid)) {\n+\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n+\t\t\t\t\t \"reference already exists\"),\n+\t\t\t\t ref_update_original_update_refname(u));\n+\t\t\treturn TRANSACTION_CREATE_EXISTS;\n+\t\t}\n+\t\telse if (is_null_oid(¤t_oid))\n+\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n+\t\t\t\t\t \"reference is missing but expected %s\"),\n+\t\t\t\t ref_update_original_update_refname(u),\n+\t\t\t\t oid_to_hex(&u->old_oid));\n+\t\telse\n+\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n+\t\t\t\t\t \"is at %s but expected %s\"),\n+\t\t\t\t ref_update_original_update_refname(u),\n+\t\t\t\t oid_to_hex(¤t_oid),\n+\t\t\t\t oid_to_hex(&u->old_oid));\n+\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t}\n+\n+\t/*\n+\t * If all of the following conditions are true:\n+\t *\n+\t * - We're not about to write a symref.\n+\t * - We're not about to write a log-only entry.\n+\t * - Old and new object ID are different.\n+\t *\n+\t * Then we're essentially doing a no-op update that can be\n+\t * skipped. This is not only for the sake of efficiency, but\n+\t * also skips writing unneeded reflog entries.\n+\t */\n+\tif ((u->type & REF_ISSYMREF) ||\n+\t (u->flags & REF_LOG_ONLY) ||\n+\t (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid)))\n+\t\treturn queue_transaction_update(refs, tx_data, u,\n+\t\t\t\t\t ¤t_oid, err);\n+\n+\treturn 0;\n+}\n+\n static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\t\t struct ref_transaction *transaction,\n \t\t\t\t\t struct strbuf *err)\n@@ -1124,239 +1360,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \tret = 0;\n \n \tfor (i = 0; i < transaction->nr; i++) {\n-\t\tstruct ref_update *u = transaction->updates[i];\n-\t\tstruct object_id current_oid = {0};\n-\t\tconst char *rewritten_ref;\n-\n-\t\t/*\n-\t\t * There is no need to reload the respective backends here as\n-\t\t * we have already reloaded them when preparing the transaction\n-\t\t * update. And given that the stacks have been locked there\n-\t\t * shouldn't have been any concurrent modifications of the\n-\t\t * stack.\n-\t\t */\n-\t\tret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);\n+\t\tret = prepare_single_update(ref_store, refs, tx_data,\n+\t\t\t\t\t transaction, be,\n+\t\t\t\t\t transaction->updates[i], head_type,\n+\t\t\t\t\t &head_referent, &referent, err);\n \t\tif (ret)\n \t\t\tgoto done;\n-\n-\t\t/* Verify that the new object ID is valid. */\n-\t\tif ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&\n-\t\t !(u->flags & REF_SKIP_OID_VERIFICATION) &&\n-\t\t !(u->flags & REF_LOG_ONLY)) {\n-\t\t\tstruct object *o = parse_object(refs->base.repo, &u->new_oid);\n-\t\t\tif (!o) {\n-\t\t\t\tstrbuf_addf(err,\n-\t\t\t\t\t _(\"trying to write ref '%s' with nonexistent object %s\"),\n-\t\t\t\t\t u->refname, oid_to_hex(&u->new_oid));\n-\t\t\t\tret = -1;\n-\t\t\t\tgoto done;\n-\t\t\t}\n-\n-\t\t\tif (o->type != OBJ_COMMIT && is_branch(u->refname)) {\n-\t\t\t\tstrbuf_addf(err, _(\"trying to write non-commit object %s to branch '%s'\"),\n-\t\t\t\t\t oid_to_hex(&u->new_oid), u->refname);\n-\t\t\t\tret = -1;\n-\t\t\t\tgoto done;\n-\t\t\t}\n-\t\t}\n-\n-\t\t/*\n-\t\t * When we update the reference that HEAD points to we enqueue\n-\t\t * a second log-only update for HEAD so that its reflog is\n-\t\t * updated accordingly.\n-\t\t */\n-\t\tif (head_type == REF_ISSYMREF &&\n-\t\t !(u->flags & REF_LOG_ONLY) &&\n-\t\t !(u->flags & REF_UPDATE_VIA_HEAD) &&\n-\t\t !strcmp(rewritten_ref, head_referent.buf)) {\n-\t\t\t/*\n-\t\t\t * First make sure that HEAD is not already in the\n-\t\t\t * transaction. This check is O(lg N) in the transaction\n-\t\t\t * size, but it happens at most once per transaction.\n-\t\t\t */\n-\t\t\tif (string_list_has_string(&transaction->refnames, \"HEAD\")) {\n-\t\t\t\t/* An entry already existed */\n-\t\t\t\tstrbuf_addf(err,\n-\t\t\t\t\t _(\"multiple updates for 'HEAD' (including one \"\n-\t\t\t\t\t \"via its referent '%s') are not allowed\"),\n-\t\t\t\t\t u->refname);\n-\t\t\t\tret = TRANSACTION_NAME_CONFLICT;\n-\t\t\t\tgoto done;\n-\t\t\t}\n-\n-\t\t\tref_transaction_add_update(\n-\t\t\t\ttransaction, \"HEAD\",\n-\t\t\t\tu->flags | REF_LOG_ONLY | REF_NO_DEREF,\n-\t\t\t\t&u->new_oid, &u->old_oid, NULL, NULL, NULL,\n-\t\t\t\tu->msg);\n-\t\t}\n-\n-\t\tret = reftable_backend_read_ref(be, rewritten_ref,\n-\t\t\t\t\t\t¤t_oid, &referent, &u->type);\n-\t\tif (ret < 0)\n-\t\t\tgoto done;\n-\t\tif (ret > 0 && !ref_update_expects_existing_old_ref(u)) {\n-\t\t\t/*\n-\t\t\t * The reference does not exist, and we either have no\n-\t\t\t * old object ID or expect the reference to not exist.\n-\t\t\t * We can thus skip below safety checks as well as the\n-\t\t\t * symref splitting. But we do want to verify that\n-\t\t\t * there is no conflicting reference here so that we\n-\t\t\t * can output a proper error message instead of failing\n-\t\t\t * at a later point.\n-\t\t\t */\n-\t\t\tret = refs_verify_refname_available(ref_store, u->refname,\n-\t\t\t\t\t\t\t &transaction->refnames, NULL,\n-\t\t\t\t\t\t\t transaction->flags & REF_TRANSACTION_FLAG_INITIAL,\n-\t\t\t\t\t\t\t err);\n-\t\t\tif (ret < 0)\n-\t\t\t\tgoto done;\n-\n-\t\t\t/*\n-\t\t\t * There is no need to write the reference deletion\n-\t\t\t * when the reference in question doesn't exist.\n-\t\t\t */\n-\t\t\t if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {\n-\t\t\t\t ret = queue_transaction_update(refs, tx_data, u,\n-\t\t\t\t\t\t\t\t¤t_oid, err);\n-\t\t\t\t if (ret)\n-\t\t\t\t\t goto done;\n-\t\t\t }\n-\n-\t\t\tcontinue;\n-\t\t}\n-\t\tif (ret > 0) {\n-\t\t\t/* The reference does not exist, but we expected it to. */\n-\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n-\t\t\t\t \"unable to resolve reference '%s'\"),\n-\t\t\t\t ref_update_original_update_refname(u), u->refname);\n-\t\t\tret = -1;\n-\t\t\tgoto done;\n-\t\t}\n-\n-\t\tif (u->type & REF_ISSYMREF) {\n-\t\t\t/*\n-\t\t\t * The reftable stack is locked at this point already,\n-\t\t\t * so it is safe to call `refs_resolve_ref_unsafe()`\n-\t\t\t * here without causing races.\n-\t\t\t */\n-\t\t\tconst char *resolved = refs_resolve_ref_unsafe(&refs->base, u->refname, 0,\n-\t\t\t\t\t\t\t\t ¤t_oid, NULL);\n-\n-\t\t\tif (u->flags & REF_NO_DEREF) {\n-\t\t\t\tif (u->flags & REF_HAVE_OLD && !resolved) {\n-\t\t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n-\t\t\t\t\t\t \"error reading reference\"), u->refname);\n-\t\t\t\t\tret = -1;\n-\t\t\t\t\tgoto done;\n-\t\t\t\t}\n-\t\t\t} else {\n-\t\t\t\tstruct ref_update *new_update;\n-\t\t\t\tint new_flags;\n-\n-\t\t\t\tnew_flags = u->flags;\n-\t\t\t\tif (!strcmp(rewritten_ref, \"HEAD\"))\n-\t\t\t\t\tnew_flags |= REF_UPDATE_VIA_HEAD;\n-\n-\t\t\t\tif (string_list_has_string(&transaction->refnames, referent.buf)) {\n-\t\t\t\t\tstrbuf_addf(err,\n-\t\t\t\t\t\t _(\"multiple updates for '%s' (including one \"\n-\t\t\t\t\t\t \"via symref '%s') are not allowed\"),\n-\t\t\t\t\t\t referent.buf, u->refname);\n-\t\t\t\t\tret = TRANSACTION_NAME_CONFLICT;\n-\t\t\t\t\tgoto done;\n-\t\t\t\t}\n-\n-\t\t\t\t/*\n-\t\t\t\t * If we are updating a symref (eg. HEAD), we should also\n-\t\t\t\t * update the branch that the symref points to.\n-\t\t\t\t *\n-\t\t\t\t * This is generic functionality, and would be better\n-\t\t\t\t * done in refs.c, but the current implementation is\n-\t\t\t\t * intertwined with the locking in files-backend.c.\n-\t\t\t\t */\n-\t\t\t\tnew_update = ref_transaction_add_update(\n-\t\t\t\t\ttransaction, referent.buf, new_flags,\n-\t\t\t\t\tu->new_target ? NULL : &u->new_oid,\n-\t\t\t\t\tu->old_target ? NULL : &u->old_oid,\n-\t\t\t\t\tu->new_target, u->old_target,\n-\t\t\t\t\tu->committer_info, u->msg);\n-\n-\t\t\t\tnew_update->parent_update = u;\n-\n-\t\t\t\t/*\n-\t\t\t\t * Change the symbolic ref update to log only. Also, it\n-\t\t\t\t * doesn't need to check its old OID value, as that will be\n-\t\t\t\t * done when new_update is processed.\n-\t\t\t\t */\n-\t\t\t\tu->flags |= REF_LOG_ONLY | REF_NO_DEREF;\n-\t\t\t\tu->flags &= ~REF_HAVE_OLD;\n-\t\t\t}\n-\t\t}\n-\n-\t\t/*\n-\t\t * Verify that the old object matches our expectations. Note\n-\t\t * that the error messages here do not make a lot of sense in\n-\t\t * the context of the reftable backend as we never lock\n-\t\t * individual refs. But the error messages match what the files\n-\t\t * backend returns, which keeps our tests happy.\n-\t\t */\n-\t\tif (u->old_target) {\n-\t\t\tif (!(u->type & REF_ISSYMREF)) {\n-\t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n-\t\t\t\t\t \"expected symref with target '%s': \"\n-\t\t\t\t\t \"but is a regular ref\"),\n-\t\t\t\t\t ref_update_original_update_refname(u),\n-\t\t\t\t\t u->old_target);\n-\t\t\t\tret = -1;\n-\t\t\t\tgoto done;\n-\t\t\t}\n-\n-\t\t\tif (ref_update_check_old_target(referent.buf, u, err)) {\n-\t\t\t\tret = -1;\n-\t\t\t\tgoto done;\n-\t\t\t}\n-\t\t} else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {\n-\t\t\tret = TRANSACTION_NAME_CONFLICT;\n-\t\t\tif (is_null_oid(&u->old_oid)) {\n-\t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n-\t\t\t\t\t\t \"reference already exists\"),\n-\t\t\t\t\t ref_update_original_update_refname(u));\n-\t\t\t\tret = TRANSACTION_CREATE_EXISTS;\n-\t\t\t}\n-\t\t\telse if (is_null_oid(¤t_oid))\n-\t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n-\t\t\t\t\t\t \"reference is missing but expected %s\"),\n-\t\t\t\t\t ref_update_original_update_refname(u),\n-\t\t\t\t\t oid_to_hex(&u->old_oid));\n-\t\t\telse\n-\t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n-\t\t\t\t\t\t \"is at %s but expected %s\"),\n-\t\t\t\t\t ref_update_original_update_refname(u),\n-\t\t\t\t\t oid_to_hex(¤t_oid),\n-\t\t\t\t\t oid_to_hex(&u->old_oid));\n-\t\t\tgoto done;\n-\t\t}\n-\n-\t\t/*\n-\t\t * If all of the following conditions are true:\n-\t\t *\n-\t\t * - We're not about to write a symref.\n-\t\t * - We're not about to write a log-only entry.\n-\t\t * - Old and new object ID are different.\n-\t\t *\n-\t\t * Then we're essentially doing a no-op update that can be\n-\t\t * skipped. This is not only for the sake of efficiency, but\n-\t\t * also skips writing unneeded reflog entries.\n-\t\t */\n-\t\tif ((u->type & REF_ISSYMREF) ||\n-\t\t (u->flags & REF_LOG_ONLY) ||\n-\t\t (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid))) {\n-\t\t\tret = queue_transaction_update(refs, tx_data, u,\n-\t\t\t\t\t\t ¤t_oid, err);\n-\t\t\tif (ret)\n-\t\t\t\tgoto done;\n-\t\t}\n \t}\n \n \ttransaction->backend_data = tx_data;\n\n-- \n2.47.0\n\n"},{"id":"512043","messageId":"20250207-245-partially-atomic-ref-updates-v1-5-e6a3690ff23a@gmail.com","threadId":"62915","inReplyTo":"20250207-245-partially-atomic-ref-updates-v1-0-e6a3690ff23a@gmail.com","subject":"[PATCH 5/6] refs: implement partial reference transaction support","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-02-07T07:34:40Z","receivedAt":"2025-02-07T07:37:15Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Git's reference transactions are all-or-nothing: either all updates\nsucceed, or none do. While this atomic behavior is generally desirable,\nit can be suboptimal when using the reftable backend, where batching\nmultiple reference updates into a single transaction is more efficient\nthan performing them sequentially.\n\nIntroduce partial transaction support through a new flag\n`REF_TRANSACTION_ALLOW_PARTIAL`. When this flag is set, individual\nreference updates that would normally fail the entire transaction are\ninstead marked as rejected while allowing other updates to proceed. This\nprovides more flexibility while maintaining transactional integrity\nwhere needed.\n\nThe implementation introduces several key components:\n\n - Add 'rejected' and 'rejection_err' fields to struct `ref_update` to\n track failed updates and their failure reasons.\n\n - Modify reference backends (files, packed, reftable) to handle\n partial transactions by using `ref_transaction_add_rejection()`\n instead of failing the entire transaction when\n `REF_TRANSACTION_ALLOW_PARTIAL` is set.\n\n - Add `ref_transaction_for_each_rejected_update()` to let callers\n examine which updates were rejected and why.\n\nThis foundational change enables partial transaction support throughout\nthe reference subsystem. The next commit will expose this capability to\nusers by adding a `--allow-partial` flag to 'git-update-ref(1)',\nproviding both a user-facing feature and a testable implementation.\n\nSigned-off-by: Karthik Nayak \n---\n refs.c | 32 ++++++++++++++++++++++++++++++++\n refs.h | 22 ++++++++++++++++++++++\n refs/files-backend.c | 12 +++++++++++-\n refs/packed-backend.c | 26 ++++++++++++++++++++++++--\n refs/refs-internal.h | 15 +++++++++++++++\n refs/reftable-backend.c | 12 +++++++++++-\n 6 files changed, 115 insertions(+), 4 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex b420a120102b3793168598b885bba68e4f5f5f03..75dbd84acbc41658d4b8b6b5e7763c04e78d0061 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1204,6 +1204,7 @@ void ref_transaction_free(struct ref_transaction *transaction)\n \t\tfree(transaction->updates[i]->committer_info);\n \t\tfree((char *)transaction->updates[i]->new_target);\n \t\tfree((char *)transaction->updates[i]->old_target);\n+\t\tstrbuf_release(&transaction->updates[i]->rejection_err);\n \t\tfree(transaction->updates[i]);\n \t}\n \tstring_list_clear(&transaction->refnames, 0);\n@@ -1211,6 +1212,14 @@ void ref_transaction_free(struct ref_transaction *transaction)\n \tfree(transaction);\n }\n \n+void ref_transaction_add_rejection(struct ref_transaction *transaction,\n+\t\t\t\t size_t update_idx, struct strbuf *err)\n+{\n+\tstruct ref_update *update = transaction->updates[update_idx];\n+\tupdate->rejected = 1;\n+\tstrbuf_addbuf(&update->rejection_err, err);\n+}\n+\n struct ref_update *ref_transaction_add_update(\n \t\tstruct ref_transaction *transaction,\n \t\tconst char *refname, unsigned int flags,\n@@ -1237,6 +1246,8 @@ struct ref_update *ref_transaction_add_update(\n \n \tupdate->flags = flags;\n \n+\tstrbuf_init(&update->rejection_err, 0);\n+\n \tupdate->new_target = xstrdup_or_null(new_target);\n \tupdate->old_target = xstrdup_or_null(old_target);\n \tif ((flags & REF_HAVE_NEW) && new_oid)\n@@ -2676,6 +2687,27 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,\n \t}\n }\n \n+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,\n+\t\t\t\t\t ref_transaction_for_each_rejected_update_fn cb,\n+\t\t\t\t\t void *cb_data)\n+{\n+\tif (!(transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL))\n+\t\treturn;\n+\n+\tfor (size_t i = 0; i < transaction->nr; i++) {\n+\t\tstruct ref_update *update = transaction->updates[i];\n+\n+\t\tif (!update->rejected)\n+\t\t\tcontinue;\n+\n+\t\tcb(update->refname,\n+\t\t (update->flags & REF_HAVE_OLD) ? &update->old_oid : NULL,\n+\t\t (update->flags & REF_HAVE_NEW) ? &update->new_oid : NULL,\n+\t\t update->old_target, update->new_target,\n+\t\t &update->rejection_err, cb_data);\n+\t}\n+}\n+\n int refs_delete_refs(struct ref_store *refs, const char *logmsg,\n \t\t struct string_list *refnames, unsigned int flags)\n {\ndiff --git a/refs.h b/refs.h\nindex a0cdd99250e8286b55808b697b0a94afac5d8319..a0f15fdea024527fcfdb478f78cbf6fd6568a25b 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -638,6 +638,13 @@ enum ref_transaction_flag {\n \t * either be absent or null_oid.\n \t */\n \tREF_TRANSACTION_FLAG_INITIAL = (1 << 0),\n+\n+\t/*\n+\t * The transaction mechanism by default fails all updates if any conflict\n+\t * is detected. This flag allows transactions to partially apply updates\n+\t * while rejecting updates which do not match the expected state.\n+\t */\n+\tREF_TRANSACTION_ALLOW_PARTIAL = (1 << 1),\n };\n \n /*\n@@ -889,6 +896,21 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,\n \t\t\t\t\t ref_transaction_for_each_queued_update_fn cb,\n \t\t\t\t\t void *cb_data);\n \n+/*\n+ * Execute the given callback function for each of the reference updates which\n+ * have been rejected in the given transaction.\n+ */\n+typedef void ref_transaction_for_each_rejected_update_fn(const char *refname,\n+\t\t\t\t\t\t\t const struct object_id *old_oid,\n+\t\t\t\t\t\t\t const struct object_id *new_oid,\n+\t\t\t\t\t\t\t const char *old_target,\n+\t\t\t\t\t\t\t const char *new_target,\n+\t\t\t\t\t\t\t const struct strbuf *reason,\n+\t\t\t\t\t\t\t void *cb_data);\n+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,\n+\t\t\t\t\t ref_transaction_for_each_rejected_update_fn cb,\n+\t\t\t\t\t void *cb_data);\n+\n /*\n * Free `*transaction` and all associated data.\n */\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 9fc5454678340dd7c72539bfa0f15ee7eb24b1ff..99ec29164fbd30635125cc2325aab3d300cf906c 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -2852,8 +2852,18 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \n \t\tret = lock_ref_for_update(refs, update, transaction,\n \t\t\t\t\t head_ref, err);\n-\t\tif (ret)\n+\t\tif (ret) {\n+\t\t\tif (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {\n+\t\t\t\tref_transaction_add_rejection(transaction, i, err);\n+\n+\t\t\t\tstrbuf_setlen(err, 0);\n+\t\t\t\tret = 0;\n+\n+\t\t\t\tcontinue;\n+\t\t\t}\n \t\t\tgoto cleanup;\n+\t\t}\n+\n \n \t\tif (update->flags & REF_DELETING &&\n \t\t !(update->flags & REF_LOG_ONLY) &&\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 6e7acb077e81435715a1ca3cc928550147c8c56a..cb9b6f0a620eaa59941f6fbc653600304f2bae8c 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1313,9 +1313,10 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,\n * remain locked when it is done.\n */\n static int write_with_updates(struct packed_ref_store *refs,\n-\t\t\t struct string_list *updates,\n+\t\t\t struct ref_transaction *transaction,\n \t\t\t struct strbuf *err)\n {\n+\tstruct string_list *updates = &transaction->refnames;\n \tstruct ref_iterator *iter = NULL;\n \tsize_t i;\n \tint ok;\n@@ -1393,6 +1394,13 @@ static int write_with_updates(struct packed_ref_store *refs,\n \t\t\t\t\tstrbuf_addf(err, \"cannot update ref '%s': \"\n \t\t\t\t\t\t \"reference already exists\",\n \t\t\t\t\t\t update->refname);\n+\n+\t\t\t\t\tif (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {\n+\t\t\t\t\t\tref_transaction_add_rejection(transaction, i, err);\n+\t\t\t\t\t\tstrbuf_setlen(err, 0);\n+\t\t\t\t\t\tcontinue;\n+\t\t\t\t\t}\n+\n \t\t\t\t\tgoto error;\n \t\t\t\t} else if (!oideq(&update->old_oid, iter->oid)) {\n \t\t\t\t\tstrbuf_addf(err, \"cannot update ref '%s': \"\n@@ -1400,6 +1408,13 @@ static int write_with_updates(struct packed_ref_store *refs,\n \t\t\t\t\t\t update->refname,\n \t\t\t\t\t\t oid_to_hex(iter->oid),\n \t\t\t\t\t\t oid_to_hex(&update->old_oid));\n+\n+\t\t\t\t\tif (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {\n+\t\t\t\t\t\tref_transaction_add_rejection(transaction, i, err);\n+\t\t\t\t\t\tstrbuf_setlen(err, 0);\n+\t\t\t\t\t\tcontinue;\n+\t\t\t\t\t}\n+\n \t\t\t\t\tgoto error;\n \t\t\t\t}\n \t\t\t}\n@@ -1434,6 +1449,13 @@ static int write_with_updates(struct packed_ref_store *refs,\n \t\t\t\t\t \"reference is missing but expected %s\",\n \t\t\t\t\t update->refname,\n \t\t\t\t\t oid_to_hex(&update->old_oid));\n+\n+\t\t\t\tif (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {\n+\t\t\t\t\tref_transaction_add_rejection(transaction, i, err);\n+\t\t\t\t\tstrbuf_setlen(err, 0);\n+\t\t\t\t\tcontinue;\n+\t\t\t\t}\n+\n \t\t\t\tgoto error;\n \t\t\t}\n \t\t}\n@@ -1657,7 +1679,7 @@ static int packed_transaction_prepare(struct ref_store *ref_store,\n \t\tdata->own_lock = 1;\n \t}\n \n-\tif (write_with_updates(refs, &transaction->refnames, err))\n+\tif (write_with_updates(refs, transaction, err))\n \t\tgoto failure;\n \n \ttransaction->state = REF_TRANSACTION_PREPARED;\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 434362b6099a35f92906a04ddd65365140147572..6b8f5b2bd83baa22480083e1002daba9300f1b70 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -3,6 +3,7 @@\n \n #include \"refs.h\"\n #include \"iterator.h\"\n+#include \"strbuf.h\"\n #include \"string-list.h\"\n \n struct fsck_options;\n@@ -123,6 +124,13 @@ struct ref_update {\n \t */\n \tunsigned int index;\n \n+\t/*\n+\t * Used in partial transactions to mark a given update as rejected,\n+\t * with rejection reason.\n+\t */\n+\tunsigned int rejected;\n+\tstruct strbuf rejection_err;\n+\n \t/*\n \t * If this ref_update was split off of a symref update via\n \t * split_symref_update(), then this member points at that\n@@ -142,6 +150,13 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t struct object_id *oid, struct strbuf *referent,\n \t\t unsigned int *type, int *failure_errno);\n \n+/*\n+ * Mark a given update as rejected with a given reason. To be used in conjuction\n+ * with the `REF_TRANSACTION_ALLOW_PARTIAL` flag to allow partial transactions.\n+ */\n+void ref_transaction_add_rejection(struct ref_transaction *transaction,\n+\t\t\t\t size_t update_idx, struct strbuf *err);\n+\n /*\n * Add a ref_update with the specified properties to transaction, and\n * return a pointer to the new object. This function does not verify\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex 5533acfaf9027765d5a270abfce96225e42cc823..a2d86d1c5098b30bd212fc12a3708d2c0a60c677 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -1364,8 +1364,18 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\t\t transaction, be,\n \t\t\t\t\t transaction->updates[i], head_type,\n \t\t\t\t\t &head_referent, &referent, err);\n-\t\tif (ret)\n+\n+\t\tif (ret) {\n+\t\t\tif (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {\n+\t\t\t\tref_transaction_add_rejection(transaction, i, err);\n+\n+\t\t\t\tstrbuf_setlen(err, 0);\n+\t\t\t\tret = 0;\n+\n+\t\t\t\tcontinue;\n+\t\t\t}\n \t\t\tgoto done;\n+\t\t}\n \t}\n \n \ttransaction->backend_data = tx_data;\n\n-- \n2.47.0\n\n"},{"id":"512044","messageId":"20250207-245-partially-atomic-ref-updates-v1-6-e6a3690ff23a@gmail.com","threadId":"62915","inReplyTo":"20250207-245-partially-atomic-ref-updates-v1-0-e6a3690ff23a@gmail.com","subject":"[PATCH 6/6] update-ref: add --allow-partial flag for stdin mode","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-02-07T07:34:41Z","receivedAt":"2025-02-07T07:37:18Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"When updating multiple references through stdin, Git's update-ref\ncommand normally aborts the entire transaction if any single update\nfails. While this atomic behavior prevents partial updates by default,\nthere are cases where applying successful updates while reporting\nfailures is desirable.\n\nAdd a new `--allow-partial` flag that allows the transaction to continue\neven when individual reference updates fail. This flag can only be used\nin `--stdin` mode and builds upon the partial transaction support added\nto the refs subsystem. When enabled, failed updates are reported in the\nfollowing format:\n\n rejected SP ( | ) SP ( | ) SP LF\n\nor with `-z`:\n\n rejected NUL ( | ) NUL ( | ) NUL NUL\n\nUpdate the documentation to reflect this change and also tests to cover\ndifferent scenarios where an update could be rejected.\n\nSigned-off-by: Karthik Nayak \n---\n Documentation/git-update-ref.txt | 12 ++-\n builtin/update-ref.c | 53 +++++++++--\n t/t1400-update-ref.sh | 191 +++++++++++++++++++++++++++++++++++++++\n 3 files changed, 248 insertions(+), 8 deletions(-)\n\ndiff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt\nindex 9e6935d38d031b4890135e0cce36fffcc349ac1d..529d3c15404cdc13216219fba6f56dde91f4909c 100644\n--- a/Documentation/git-update-ref.txt\n+++ b/Documentation/git-update-ref.txt\n@@ -8,7 +8,7 @@ git-update-ref - Update the object name stored in a ref safely\n SYNOPSIS\n --------\n [verse]\n-'git update-ref' [-m ] [--no-deref] (-d [] | [--create-reflog] [] | --stdin [-z])\n+'git update-ref' [-m ] [--no-deref] (-d [] | [--create-reflog] [] | --stdin [-z] [--allow-partial])\n \n DESCRIPTION\n -----------\n@@ -57,6 +57,12 @@ performs all modifications together. Specify commands of the form:\n With `--create-reflog`, update-ref will create a reflog for each ref\n even if one would not ordinarily be created.\n \n+With `--allow-partial`, update-ref will process the transaction even if\n+some of the updates fail, allowing remaining updates to be applied.\n+Failed updates will be printed in the following format:\n+\n+\trejected SP ( | ) SP ( | ) SP LF\n+\n Quote fields containing whitespace as if they were strings in C source\n code; i.e., surrounded by double-quotes and with backslash escapes.\n Use 40 \"0\" characters or the empty string to specify a zero value. To\n@@ -82,6 +88,10 @@ quoting:\n In this format, use 40 \"0\" to specify a zero value, and use the empty\n string to specify a missing value.\n \n+With `-z`, `--allow-partial` will print rejections in the following form:\n+\n+\trejected NUL ( | ) NUL ( | ) NUL NUL\n+\n In either format, values can be specified in any form that Git\n recognizes as an object name. Commands in any other format or a\n repeated produce an error. Command meanings are:\ndiff --git a/builtin/update-ref.c b/builtin/update-ref.c\nindex 4d35bdc4b4b57937112e6c4c9740420b1f1771e5..83dcb7d8d73f423226c36b61374c86c6b29ec756 100644\n--- a/builtin/update-ref.c\n+++ b/builtin/update-ref.c\n@@ -5,6 +5,7 @@\n #include \"config.h\"\n #include \"gettext.h\"\n #include \"hash.h\"\n+#include \"hex.h\"\n #include \"refs.h\"\n #include \"object-name.h\"\n #include \"parse-options.h\"\n@@ -13,7 +14,7 @@\n static const char * const git_update_ref_usage[] = {\n \tN_(\"git update-ref [] -d []\"),\n \tN_(\"git update-ref [] []\"),\n-\tN_(\"git update-ref [] --stdin [-z]\"),\n+\tN_(\"git update-ref [] --stdin [-z] [--allow-partial]\"),\n \tNULL\n };\n \n@@ -562,6 +563,30 @@ static void parse_cmd_abort(struct ref_transaction *transaction,\n \treport_ok(\"abort\");\n }\n \n+static void print_rejected_refs(const char *refname,\n+\t\t\t\tconst struct object_id *old_oid,\n+\t\t\t\tconst struct object_id *new_oid,\n+\t\t\t\tconst char *old_target,\n+\t\t\t\tconst char *new_target,\n+\t\t\t\tconst struct strbuf *reason,\n+\t\t\t\tvoid *cb_data UNUSED)\n+{\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tchar space = ' ';\n+\n+\tif (!line_termination)\n+\t\tspace = line_termination;\n+\n+\tstrbuf_addf(&sb, \"rejected%c%s%c%s%c%c%s%c%s%c\", space,\n+\t\t refname, space, new_oid ? oid_to_hex(new_oid) : new_target,\n+\t\t space, space, old_oid ? oid_to_hex(old_oid) : old_target,\n+\t\t space, reason->buf, line_termination);\n+\n+\tfwrite(sb.buf, sb.len, 1, stdout);\n+\tstrbuf_release(&sb);\n+\tfflush(stdout);\n+}\n+\n static void parse_cmd_commit(struct ref_transaction *transaction,\n \t\t\t const char *next, const char *end UNUSED)\n {\n@@ -570,6 +595,10 @@ static void parse_cmd_commit(struct ref_transaction *transaction,\n \t\tdie(\"commit: extra input: %s\", next);\n \tif (ref_transaction_commit(transaction, &error))\n \t\tdie(\"commit: %s\", error.buf);\n+\n+\tref_transaction_for_each_rejected_update(transaction,\n+\t\t\t\t\t\t print_rejected_refs, NULL);\n+\n \treport_ok(\"commit\");\n \tref_transaction_free(transaction);\n }\n@@ -606,7 +635,7 @@ static const struct parse_cmd {\n \t{ \"commit\", parse_cmd_commit, 0, UPDATE_REFS_CLOSED },\n };\n \n-static void update_refs_stdin(void)\n+static void update_refs_stdin(unsigned int flags)\n {\n \tstruct strbuf input = STRBUF_INIT, err = STRBUF_INIT;\n \tenum update_refs_state state = UPDATE_REFS_OPEN;\n@@ -614,7 +643,7 @@ static void update_refs_stdin(void)\n \tint i, j;\n \n \ttransaction = ref_store_transaction_begin(get_main_ref_store(the_repository),\n-\t\t\t\t\t\t 0, &err);\n+\t\t\t\t\t\t flags, &err);\n \tif (!transaction)\n \t\tdie(\"%s\", err.buf);\n \n@@ -682,7 +711,7 @@ static void update_refs_stdin(void)\n \t\t\t */\n \t\t\tstate = cmd->state;\n \t\t\ttransaction = ref_store_transaction_begin(get_main_ref_store(the_repository),\n-\t\t\t\t\t\t\t\t 0, &err);\n+\t\t\t\t\t\t\t\t flags, &err);\n \t\t\tif (!transaction)\n \t\t\t\tdie(\"%s\", err.buf);\n \n@@ -698,6 +727,8 @@ static void update_refs_stdin(void)\n \t\t/* Commit by default if no transaction was requested. */\n \t\tif (ref_transaction_commit(transaction, &err))\n \t\t\tdie(\"%s\", err.buf);\n+\t\tref_transaction_for_each_rejected_update(transaction,\n+\t\t\t\t\t\t print_rejected_refs, NULL);\n \t\tref_transaction_free(transaction);\n \t\tbreak;\n \tcase UPDATE_REFS_STARTED:\n@@ -723,7 +754,8 @@ int cmd_update_ref(int argc,\n \tconst char *refname, *oldval;\n \tstruct object_id oid, oldoid;\n \tint delete = 0, no_deref = 0, read_stdin = 0, end_null = 0;\n-\tint create_reflog = 0;\n+\tint create_reflog = 0, allow_partial = 0;\n+\n \tstruct option options[] = {\n \t\tOPT_STRING( 'm', NULL, &msg, N_(\"reason\"), N_(\"reason of the update\")),\n \t\tOPT_BOOL('d', NULL, &delete, N_(\"delete the reference\")),\n@@ -732,6 +764,7 @@ int cmd_update_ref(int argc,\n \t\tOPT_BOOL('z', NULL, &end_null, N_(\"stdin has NUL-terminated arguments\")),\n \t\tOPT_BOOL( 0 , \"stdin\", &read_stdin, N_(\"read updates from stdin\")),\n \t\tOPT_BOOL( 0 , \"create-reflog\", &create_reflog, N_(\"create a reflog\")),\n+\t\tOPT_BOOL('0', \"allow-partial\", &allow_partial, N_(\"allow partial transactions\")),\n \t\tOPT_END(),\n \t};\n \n@@ -749,13 +782,19 @@ int cmd_update_ref(int argc,\n \t}\n \n \tif (read_stdin) {\n+\t\tunsigned int flags = 0;\n+\n+\t\tif (allow_partial)\n+\t\t\tflags |= REF_TRANSACTION_ALLOW_PARTIAL;\n+\n \t\tif (delete || argc > 0)\n \t\t\tusage_with_options(git_update_ref_usage, options);\n \t\tif (end_null)\n \t\t\tline_termination = '\\0';\n-\t\tupdate_refs_stdin();\n+\t\tupdate_refs_stdin(flags);\n \t\treturn 0;\n-\t}\n+\t} else if (allow_partial)\n+\t\tdie(\"--allow-partial can only be used with --stdin\");\n \n \tif (end_null)\n \t\tusage_with_options(git_update_ref_usage, options);\ndiff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh\nindex 29045aad43906fce3f64fb82ee98fb5f80d4796b..4f02f1974de4164442507a2eaec258edf6574f1f 100755\n--- a/t/t1400-update-ref.sh\n+++ b/t/t1400-update-ref.sh\n@@ -2066,6 +2066,197 @@ do\n \t\tgrep \"$(git rev-parse $a) $(git rev-parse $a)\" actual\n \t'\n \n+\ttest_expect_success \"stdin $type allow-partial\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit commit &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$head\" \"$Z\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$head\" \"$Z\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --allow-partial expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\tgit rev-parse refs/heads/ref2 >actual &&\n+\t\t\ttest_cmp expect actual\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type allow-partial with invalid new_oid\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\t\t\tgit update-ref refs/heads/ref2 $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$(test_oid 001)\" \"$head\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --allow-partial stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\techo $head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref2 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"trying to write ref ${SQ}refs/heads/ref2${SQ} with nonexistent object\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type allow-partial with non-commit new_oid\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\thead_tree=$(git rev-parse HEAD^{tree}) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\t\t\tgit update-ref refs/heads/ref2 $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$head_tree\" \"$head\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --allow-partial stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\techo $head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref2 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"trying to write non-commit object $head_tree to branch ${SQ}refs/heads/ref2${SQ}\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type allow-partial with non-existent ref\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$old_head\" \"$head\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --allow-partial stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_must_fail git rev-parse refs/heads/ref2 &&\n+\t\t\ttest_grep -q \"unable to resolve reference\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type allow-partial with dangling symref\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\t\t\tgit symbolic-ref refs/heads/ref2 refs/heads/nonexistent &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$old_head\" \"$head\" >>stdin &&\n+\t\t\tgit update-ref $type --no-deref --stdin --allow-partial stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\techo $head >expect &&\n+\t\t\ttest_must_fail git rev-parse refs/heads/ref2 &&\n+\t\t\ttest_grep -q \"reference is missing but expected $head\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type allow-partial with regular ref as symref\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\t\t\tgit update-ref refs/heads/ref2 $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"symref-update refs/heads/ref2\" \"$old_head\" \"ref\" \"refs/heads/nonexistent\" >>stdin &&\n+\t\t\tgit update-ref $type --no-deref --stdin --allow-partial stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\techo $head >expect &&\n+\t\t\techo $head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref2 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"expected symref with target ${SQ}refs/heads/nonexistent${SQ}: but is a regular ref\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type allow-partial with invalid old_oid\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\t\t\tgit update-ref refs/heads/ref2 $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$old_head\" \"$Z\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --allow-partial stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\techo $head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref2 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"reference already exists\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type allow-partial with incorrect old oid\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\t\t\tgit update-ref refs/heads/ref2 $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$head\" \"$old_head\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --allow-partial stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\techo $head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref2 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"${SQ}refs/heads/ref2${SQ}: is at $head but expected $old_head\" stdout\n+\t\t)\n+\t'\n done\n \n test_expect_success 'update-ref should also create reflog for HEAD' '\n\n-- \n2.47.0\n\n"},{"id":"512102","messageId":"Z6Yw9RK6JRKERmn9@pks.im","threadId":"62915","inReplyTo":"20250207-245-partially-atomic-ref-updates-v1-1-e6a3690ff23a@gmail.com","subject":"Re: [PATCH 1/6] refs/files: remove duplicate check in `split_symref_update()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-07T16:12:37Z","receivedAt":"2025-02-07T16:12:47Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Feb 07, 2025 at 08:34:36AM +0100, Karthik Nayak wrote:\n> In split_symref_update(), there were two redundant checks:\n> - At the start: checking if refname exists in `affected_refnames`.\n> - After adding refname: checking if the item added to\n> `affected_refnames` contains the util field.\n\nOkay, it took me a bit longer to understand what's going on here. What\nyou're saying is that we already use `string_list_has_string()` at the\nstart of `split_symref_update()`, and if that returns true then we would\nbail out. Consequently, it is impossible for `string_list_insert()` to\nfind a preexisting values.\n\nMakes sense, but I think that could be explained a bit better.\n\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 29f08dced40418eb815072c6335e0c3d1a45c7d8..c6a3f6d6261a894e1c294bb1329fdf8079a39eb4 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -2846,13 +2838,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n> \t\tif (update->flags & REF_LOG_ONLY)\n> \t\t\tcontinue;\n> \n> -\t\titem = string_list_append(&affected_refnames, update->refname);\n> -\t\t/*\n> -\t\t * We store a pointer to update in item->util, but at\n> -\t\t * the moment we never use the value of this field\n> -\t\t * except to check whether it is non-NULL.\n> -\t\t */\n> -\t\titem->util = update;\n> +\t\tstring_list_append(&affected_refnames, update->refname);\n\n\nNice to see this and other code removed.\n\nPatrick\n"},{"id":"512103","messageId":"Z6Yw_f6PGJ2X_Iue@pks.im","threadId":"62915","inReplyTo":"20250207-245-partially-atomic-ref-updates-v1-2-e6a3690ff23a@gmail.com","subject":"Re: [PATCH 2/6] refs: move duplicate refname update check to generic layer","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-07T16:12:45Z","receivedAt":"2025-02-07T16:12:48Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Feb 07, 2025 at 08:34:37AM +0100, Karthik Nayak wrote:\n> Move the tracking of refnames in `affected_refnames` from individual\n> backends into the generic layer in 'refs.c'. This centralizes the\n> duplicate refname detection that was previously handled separately by\n> each backend.\n\nExciting, this has been on my TODO list for quite a while already.\n\n> Make some changes to accommodate this move:\n> \n> - Add a `string_list` field `refnames` to `ref_transaction` to contain\n> all the references in a transaction. This field is updated whenever\n> a new update is added.\n> \n> - Modify the backends to use this field internally as needed. The\n> backends need to check if an update for refname already exists when\n> splitting symrefs or adding an update for 'HEAD'.\n\nOkay. Is this actually necessary to be handled by the backends? I\nwould've expected that it is possible to split up symref updates so that\nwe insert both symref and target into the list. I wouldn't be surprised\nif this wasn't easily possible though -- the logic here is surprisingly\nintricate.\n\n> - In the reftable backend, in `reftable_be_transaction_prepare()`,\n> move the instance of `string_list_has_string()` above\n> `ref_transaction_add_update()` to check before the reference is\n> added.\n> \n> This helps reduce duplication of functionality between the backends and\n> makes it easier to make changes in a more centralized manner.\n\n> Signed-off-by: Karthik Nayak \n> ---\n> refs.c | 17 ++++++++++++\n> refs/files-backend.c | 69 ++++++++++---------------------------------------\n> refs/packed-backend.c | 25 +-----------------\n> refs/refs-internal.h | 2 ++\n> refs/reftable-backend.c | 53 ++++++++++++-------------------------\n> 5 files changed, 50 insertions(+), 116 deletions(-)\n\nNice.\n\n> diff --git a/refs.c b/refs.c\n> index f4094a326a9f88f979654b668cc9c3d27d83cb5d..4c9b706461977995be1d55e7667f7fb708fbbb76 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -1175,6 +1175,7 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,\n> \tCALLOC_ARRAY(tr, 1);\n> \ttr->ref_store = refs;\n> \ttr->flags = flags;\n> +\tstring_list_init_dup(&tr->refnames);\n\nDo we actually have to duplicate strings? I would've expected that we\nkeep strings alive via the `ref_update`s anyway during the transaction's\nlifetime.\n\nIt might also be interesting to check whether using a strset for this\nis more efficient. But that is certainly outside the scope of your patch\nseries and can be done at a later point. #leftoverbit\n\n> @@ -1245,6 +1248,16 @@ struct ref_update *ref_transaction_add_update(\n> \t\tupdate->msg = normalize_reflog_message(msg);\n> \t}\n> \n> +\t/*\n> +\t * This list is generally used by the backends to avoid duplicates.\n> +\t * But we do support multiple log updates for a given refname within\n> +\t * a single transaction.\n> +\t */\n> +\tif (!(update->flags & REF_LOG_ONLY)) {\n> +\t\titem = string_list_append(&transaction->refnames, refname);\n> +\t\titem->util = update;\n> +\t}\n> +\n> \treturn update;\n> }\n> @@ -2397,6 +2410,10 @@ int ref_transaction_prepare(struct ref_transaction *transaction,\n> \t\treturn -1;\n> \t}\n> \n> +\tstring_list_sort(&transaction->refnames);\n> +\tif (ref_update_reject_duplicates(&transaction->refnames, err))\n> +\t\treturn TRANSACTION_GENERIC_ERROR;\n> +\n> \tret = refs->be->transaction_prepare(refs, transaction, err);\n> \tif (ret)\n> \t\treturn ret;\n\nOkay, we keep the list unserted initially, but sort it later before\npassing it to the backends so that `string_list_has_string()` works\ncorrectly. Good.\n\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index c6a3f6d6261a894e1c294bb1329fdf8079a39eb4..18da30c3f37dc5c09f7d81a9083d6b41d0463bd5 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -2425,7 +2423,6 @@ static int split_head_update(struct ref_update *update,\n> \t */\n> \tif (strcmp(new_update->refname, \"HEAD\"))\n> \t\tBUG(\"%s unexpectedly not 'HEAD'\", new_update->refname);\n> -\tstring_list_insert(affected_refnames, new_update->refname);\n> \n> \treturn 0;\n> }\n\nPreviously we would've inserted \"HEAD\" into the list of affected\nrefnames even if it wasn't directly updated. Why don't we have to do\nthat now anymore?\n\n> @@ -2441,7 +2438,6 @@ static int split_head_update(struct ref_update *update,\n> @@ -2491,15 +2487,6 @@ static int split_symref_update(struct ref_update *update,\n> \tupdate->flags |= REF_LOG_ONLY | REF_NO_DEREF;\n> \tupdate->flags &= ~REF_HAVE_OLD;\n> \n> -\t/*\n> -\t * Add the referent. This insertion is O(N) in the transaction\n> -\t * size, but it happens at most once per symref in a\n> -\t * transaction. Make sure to add new_update->refname, which will\n> -\t * be valid as long as affected_refnames is in use, and NOT\n> -\t * referent, which might soon be freed by our caller.\n> -\t */\n> -\tstring_list_insert(affected_refnames, new_update->refname);\n> -\n> \treturn 0;\n> }\n\nSame question here, but for symref updates.\n\n> @@ -3030,13 +2995,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n> \tif (transaction->state != REF_TRANSACTION_PREPARED)\n> \t\tBUG(\"commit called for transaction that is not prepared\");\n> \n> -\t/* Fail if a refname appears more than once in the transaction: */\n> -\tfor (i = 0; i < transaction->nr; i++)\n> -\t\tif (!(transaction->updates[i]->flags & REF_LOG_ONLY))\n> -\t\t\tstring_list_append(&affected_refnames,\n> -\t\t\t\t\t transaction->updates[i]->refname);\n> -\tstring_list_sort(&affected_refnames);\n> -\tif (ref_update_reject_duplicates(&affected_refnames, err)) {\n> +\tstring_list_sort(&transaction->refnames);\n> +\tif (ref_update_reject_duplicates(&transaction->refnames, err)) {\n> \t\tret = TRANSACTION_GENERIC_ERROR;\n> \t\tgoto cleanup;\n> \t}\n\nCan't we also make this check generic for initial transactions?\n\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index a7b6f74b6e35f897f619c540cbc600bbd888bc67..6e7acb077e81435715a1ca3cc928550147c8c56a 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -1653,34 +1648,16 @@ static int packed_transaction_prepare(struct ref_store *ref_store,\n> \t */\n> \n> \tCALLOC_ARRAY(data, 1);\n> -\tstring_list_init_nodup(&data->updates);\n> \n> \ttransaction->backend_data = data;\n> \n> -\t/*\n> -\t * Stick the updates in a string list by refname so that we\n> -\t * can sort them:\n> -\t */\n> -\tfor (i = 0; i < transaction->nr; i++) {\n> -\t\tstruct ref_update *update = transaction->updates[i];\n> -\t\tstruct string_list_item *item =\n> -\t\t\tstring_list_append(&data->updates, update->refname);\n> -\n> -\t\t/* Store a pointer to update in item->util: */\n> -\t\titem->util = update;\n> -\t}\n> -\tstring_list_sort(&data->updates);\n> -\n> -\tif (ref_update_reject_duplicates(&data->updates, err))\n> -\t\tgoto failure;\n> -\n> \tif (!is_lock_file_locked(&refs->lock)) {\n> \t\tif (packed_refs_lock(ref_store, 0, err))\n> \t\t\tgoto failure;\n> \t\tdata->own_lock = 1;\n> \t}\n> \n> -\tif (write_with_updates(refs, &data->updates, err))\n> +\tif (write_with_updates(refs, &transaction->refnames, err))\n> \t\tgoto failure;\n> \n> \ttransaction->state = REF_TRANSACTION_PREPARED;\n\nThis change is a lot more straight-forward because the packed backend\ndoes not support symrefs at all. Nice.\n\n> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\n> index d39a14c5a469d7d219362e9eae4f578784d65a5b..dd2099d94948a4f23fd9f7ddc06bf3d741229eba 100644\n> --- a/refs/reftable-backend.c\n> +++ b/refs/reftable-backend.c\n> @@ -1202,12 +1184,11 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n> \t\t\t\tgoto done;\n> \t\t\t}\n> \n> -\t\t\tnew_update = ref_transaction_add_update(\n> -\t\t\t\t\ttransaction, \"HEAD\",\n> -\t\t\t\t\tu->flags | REF_LOG_ONLY | REF_NO_DEREF,\n> -\t\t\t\t\t&u->new_oid, &u->old_oid, NULL, NULL, NULL,\n> -\t\t\t\t\tu->msg);\n> -\t\t\tstring_list_insert(&affected_refnames, new_update->refname);\n> +\t\t\tref_transaction_add_update(\n> +\t\t\t\ttransaction, \"HEAD\",\n> +\t\t\t\tu->flags | REF_LOG_ONLY | REF_NO_DEREF,\n> +\t\t\t\t&u->new_oid, &u->old_oid, NULL, NULL, NULL,\n> +\t\t\t\tu->msg);\n> \t\t}\n> \n> \t\tret = reftable_backend_read_ref(be, rewritten_ref,\n\nEquivalent question as for the files backend.\n\n> @@ -1277,6 +1258,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n> \t\t\t\tif (!strcmp(rewritten_ref, \"HEAD\"))\n> \t\t\t\t\tnew_flags |= REF_UPDATE_VIA_HEAD;\n> \n> +\t\t\t\tif (string_list_has_string(&transaction->refnames, referent.buf)) {\n> +\t\t\t\t\tstrbuf_addf(err,\n> +\t\t\t\t\t\t _(\"multiple updates for '%s' (including one \"\n> +\t\t\t\t\t\t \"via symref '%s') are not allowed\"),\n> +\t\t\t\t\t\t referent.buf, u->refname);\n> +\t\t\t\t\tret = TRANSACTION_NAME_CONFLICT;\n> +\t\t\t\t\tgoto done;\n> +\t\t\t\t}\n> +\n> \t\t\t\t/*\n> \t\t\t\t * If we are updating a symref (eg. HEAD), we should also\n> \t\t\t\t * update the branch that the symref points to.\n\nThis change surprised me a bit. You mention it in the commit message,\nbut don't state a reason why you do it.\n\nPatrick\n"},{"id":"512104","messageId":"Z6YxANrjhHRUSv9a@pks.im","threadId":"62915","inReplyTo":"20250207-245-partially-atomic-ref-updates-v1-3-e6a3690ff23a@gmail.com","subject":"Re: [PATCH 3/6] refs/files: remove duplicate duplicates check","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-07T16:12:48Z","receivedAt":"2025-02-07T16:12:52Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Feb 07, 2025 at 08:34:38AM +0100, Karthik Nayak wrote:\n> Within the files reference backend's transaction's 'finish' phase, a\n> verification step is currently performed wherein the refnames list is\n> sorted and examined for multiple updates targeting the same refname.\n> \n> It has been observed that this verification is redundant, as an\n> identical check is already executed during the transaction's 'prepare'\n> stage. Since the refnames list remains unmodified following the\n> 'prepare' stage, this secondary verification can be safely eliminated.\n> \n> The duplicate check has been removed accordingly, and the\n> `ref_update_reject_duplicates()` function has been marked as static, as\n> its usage is now confined to 'refs.c'.\n\nNice, I had been wondering about this code in the preceding commit.\n\n> diff --git a/refs.c b/refs.c\n> index 4c9b706461977995be1d55e7667f7fb708fbbb76..b420a120102b3793168598b885bba68e4f5f5f03 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -2295,8 +2295,13 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,\n> \treturn ret;\n> }\n> \n> -int ref_update_reject_duplicates(struct string_list *refnames,\n> -\t\t\t\t struct strbuf *err)\n> +/*\n> + * Write an error to `err` and return a nonzero value iff the same\n> + * refname appears multiple times in `refnames`. `refnames` must be\n> + * sorted on entry to this function.\n> + */\n> +static int ref_update_reject_duplicates(struct string_list *refnames,\n> +\t\t\t\t\tstruct strbuf *err)\n> {\n> \tsize_t i, n = refnames->nr;\n> \n\nDoubly nice that this can now be static.\n\nPatrick\n"},{"id":"512105","messageId":"Z6YxA4BlhNwbeYk-@pks.im","threadId":"62915","inReplyTo":"20250207-245-partially-atomic-ref-updates-v1-5-e6a3690ff23a@gmail.com","subject":"Re: [PATCH 5/6] refs: implement partial reference transaction support","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-07T16:12:51Z","receivedAt":"2025-02-07T16:12:55Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Feb 07, 2025 at 08:34:40AM +0100, Karthik Nayak wrote:\n> Git's reference transactions are all-or-nothing: either all updates\n> succeed, or none do. While this atomic behavior is generally desirable,\n> it can be suboptimal when using the reftable backend, where batching\n> multiple reference updates into a single transaction is more efficient\n> than performing them sequentially.\n\nIn fact it's even inefficient for the \"files\" backend. The whole\nmachinery around creating a new transaction, preparing it, committing it\nand then cleaning up its state does bring a bunch of overhead with it.\nBut true, for the \"reftable\" backend it's way more impactful.\n\n> diff --git a/refs.c b/refs.c\n> index b420a120102b3793168598b885bba68e4f5f5f03..75dbd84acbc41658d4b8b6b5e7763c04e78d0061 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -1211,6 +1212,14 @@ void ref_transaction_free(struct ref_transaction *transaction)\n> \tfree(transaction);\n> }\n> \n> +void ref_transaction_add_rejection(struct ref_transaction *transaction,\n> +\t\t\t\t size_t update_idx, struct strbuf *err)\n\n\"add\" to me sounds like you're adding a new thingy to the transaction,\nbut you rather update something. How about `ref_update_set_rejected()`\nor `ref_transacton_set_rejected()`?\n\n> +{\n> +\tstruct ref_update *update = transaction->updates[update_idx];\n\nDo we want to `BUG()` in case `update_idx >= transaction->nr`?\n\n> +\tupdate->rejected = 1;\n> +\tstrbuf_addbuf(&update->rejection_err, err);\n> +}\n\nDo we really need a string as rejection error? I'd expect that the set\nof failures that lead to rejection should be rather limited, which means\nthat we could use an enum instead. This would unify the errors across\nbackends and also allows us to figure out the root cause of rejection in\nother subsystems.\n\nIf we introduced an enum, we could eventually even iterate a bit on the\nmechanism and rather trivially tell the backends which kind of failures\nare acceptable. As an example, a conflicting ref update may for example\nbe ignored and not cause failure, a conflicting path name might cause\nfailure.\n\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 9fc5454678340dd7c72539bfa0f15ee7eb24b1ff..99ec29164fbd30635125cc2325aab3d300cf906c 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -2852,8 +2852,18 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n> \n> \t\tret = lock_ref_for_update(refs, update, transaction,\n> \t\t\t\t\t head_ref, err);\n> -\t\tif (ret)\n> +\t\tif (ret) {\n\nI wonder whether we want to accept all failures. Some failures are\ncertainly benign, like for example mismatching expected OIDs or a\nconflict due to a preexisting ref that blocks the path. But other kinds\nof failures which are unexpected might be a bit more on the dangerous\nside to accept, so I think we should be careful here.\n\nThe same comment also applies to the other backends.\n\nPatrick\n"},{"id":"512106","messageId":"Z6YxB6qL8WQFZyWW@pks.im","threadId":"62915","inReplyTo":"20250207-245-partially-atomic-ref-updates-v1-6-e6a3690ff23a@gmail.com","subject":"Re: [PATCH 6/6] update-ref: add --allow-partial flag for stdin mode","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-07T16:12:55Z","receivedAt":"2025-02-07T16:12:58Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Feb 07, 2025 at 08:34:41AM +0100, Karthik Nayak wrote:\n> diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt\n> index 9e6935d38d031b4890135e0cce36fffcc349ac1d..529d3c15404cdc13216219fba6f56dde91f4909c 100644\n> --- a/Documentation/git-update-ref.txt\n> +++ b/Documentation/git-update-ref.txt\n> @@ -8,7 +8,7 @@ git-update-ref - Update the object name stored in a ref safely\n> SYNOPSIS\n> --------\n> [verse]\n> -'git update-ref' [-m ] [--no-deref] (-d [] | [--create-reflog] [] | --stdin [-z])\n> +'git update-ref' [-m ] [--no-deref] (-d [] | [--create-reflog] [] | --stdin [-z] [--allow-partial])\n\nI think it's time that we start to split this line into multiple lines :)\n\n> diff --git a/builtin/update-ref.c b/builtin/update-ref.c\n> index 4d35bdc4b4b57937112e6c4c9740420b1f1771e5..83dcb7d8d73f423226c36b61374c86c6b29ec756 100644\n> --- a/builtin/update-ref.c\n> +++ b/builtin/update-ref.c\n> @@ -562,6 +563,30 @@ static void parse_cmd_abort(struct ref_transaction *transaction,\n> \treport_ok(\"abort\");\n> }\n> \n> +static void print_rejected_refs(const char *refname,\n> +\t\t\t\tconst struct object_id *old_oid,\n> +\t\t\t\tconst struct object_id *new_oid,\n> +\t\t\t\tconst char *old_target,\n> +\t\t\t\tconst char *new_target,\n> +\t\t\t\tconst struct strbuf *reason,\n> +\t\t\t\tvoid *cb_data UNUSED)\n> +{\n> +\tstruct strbuf sb = STRBUF_INIT;\n> +\tchar space = ' ';\n> +\n> +\tif (!line_termination)\n> +\t\tspace = line_termination;\n> +\n> +\tstrbuf_addf(&sb, \"rejected%c%s%c%s%c%c%s%c%s%c\", space,\n\nWhew, that's a lot of placeholders.\n\n> @@ -723,7 +754,8 @@ int cmd_update_ref(int argc,\n> \tconst char *refname, *oldval;\n> \tstruct object_id oid, oldoid;\n> \tint delete = 0, no_deref = 0, read_stdin = 0, end_null = 0;\n> -\tint create_reflog = 0;\n> +\tint create_reflog = 0, allow_partial = 0;\n> +\n> \tstruct option options[] = {\n> \t\tOPT_STRING( 'm', NULL, &msg, N_(\"reason\"), N_(\"reason of the update\")),\n> \t\tOPT_BOOL('d', NULL, &delete, N_(\"delete the reference\")),\n> @@ -732,6 +764,7 @@ int cmd_update_ref(int argc,\n> \t\tOPT_BOOL('z', NULL, &end_null, N_(\"stdin has NUL-terminated arguments\")),\n> \t\tOPT_BOOL( 0 , \"stdin\", &read_stdin, N_(\"read updates from stdin\")),\n> \t\tOPT_BOOL( 0 , \"create-reflog\", &create_reflog, N_(\"create a reflog\")),\n> +\t\tOPT_BOOL('0', \"allow-partial\", &allow_partial, N_(\"allow partial transactions\")),\n\nYou can use `OPT_BIT()` to set a specific bit in a flags field..\n\n> @@ -749,13 +782,19 @@ int cmd_update_ref(int argc,\n> \t}\n> \n> \tif (read_stdin) {\n> +\t\tunsigned int flags = 0;\n> +\n> +\t\tif (allow_partial)\n> +\t\t\tflags |= REF_TRANSACTION_ALLOW_PARTIAL;\n> +\n> \t\tif (delete || argc > 0)\n> \t\t\tusage_with_options(git_update_ref_usage, options);\n> \t\tif (end_null)\n> \t\t\tline_termination = '\\0';\n> -\t\tupdate_refs_stdin();\n> +\t\tupdate_refs_stdin(flags);\n> \t\treturn 0;\n> -\t}\n> +\t} else if (allow_partial)\n> +\t\tdie(\"--allow-partial can only be used with --stdin\");\n> \n> \tif (end_null)\n> \t\tusage_with_options(git_update_ref_usage, options);\n\nThe implementation is quite simple, nice.\n\nPatrick\n"},{"id":"512209","messageId":"CAOLa=ZRYvLAEbaxMf=P2euh+QO00W5RAhDTF1v4YYj29yH0YHQ@mail.gmail.com","threadId":"62915","inReplyTo":"Z6Yw9RK6JRKERmn9@pks.im","subject":"Re: [PATCH 1/6] refs/files: remove duplicate check in `split_symref_update()`","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-02-11T06:35:59Z","receivedAt":"2025-02-11T06:36:01Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> On Fri, Feb 07, 2025 at 08:34:36AM +0100, Karthik Nayak wrote:\n>> In split_symref_update(), there were two redundant checks:\n>> - At the start: checking if refname exists in `affected_refnames`.\n>> - After adding refname: checking if the item added to\n>> `affected_refnames` contains the util field.\n>\n> Okay, it took me a bit longer to understand what's going on here. What\n> you're saying is that we already use `string_list_has_string()` at the\n> start of `split_symref_update()`, and if that returns true then we would\n> bail out. Consequently, it is impossible for `string_list_insert()` to\n> find a preexisting values.\n>\n> Makes sense, but I think that could be explained a bit better.\n>\n\nThat's correct.\n\nI'll rewrite it to make it clearer. Thanks.\n\n>> diff --git a/refs/files-backend.c b/refs/files-backend.c\n>> index 29f08dced40418eb815072c6335e0c3d1a45c7d8..c6a3f6d6261a894e1c294bb1329fdf8079a39eb4 100644\n>> --- a/refs/files-backend.c\n>> +++ b/refs/files-backend.c\n>> @@ -2846,13 +2838,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n>> \t\tif (update->flags & REF_LOG_ONLY)\n>> \t\t\tcontinue;\n>>\n>> -\t\titem = string_list_append(&affected_refnames, update->refname);\n>> -\t\t/*\n>> -\t\t * We store a pointer to update in item->util, but at\n>> -\t\t * the moment we never use the value of this field\n>> -\t\t * except to check whether it is non-NULL.\n>> -\t\t */\n>> -\t\titem->util = update;\n>> +\t\tstring_list_append(&affected_refnames, update->refname);\n>\n>\n> Nice to see this and other code removed.\n>\n> Patrick\n"},{"id":"512227","messageId":"CAOLa=ZT+uFOCDR1nJpOoCneBVT=fKDLMhmSwVHJxVOPeaX3W2w@mail.gmail.com","threadId":"62915","inReplyTo":"Z6Yw_f6PGJ2X_Iue@pks.im","subject":"Re: [PATCH 2/6] refs: move duplicate refname update check to generic layer","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-02-11T10:33:25Z","receivedAt":"2025-02-11T10:33:27Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> On Fri, Feb 07, 2025 at 08:34:37AM +0100, Karthik Nayak wrote:\n>> Move the tracking of refnames in `affected_refnames` from individual\n>> backends into the generic layer in 'refs.c'. This centralizes the\n>> duplicate refname detection that was previously handled separately by\n>> each backend.\n>\n> Exciting, this has been on my TODO list for quite a while already.\n>\n\nYeah, I saw that you left a TODO in the reftable backend too. This\nchange was not really needed for partial transactions. But it does make\nthings a bit nicer and easier.\n\n>> Make some changes to accommodate this move:\n>>\n>> - Add a `string_list` field `refnames` to `ref_transaction` to contain\n>> all the references in a transaction. This field is updated whenever\n>> a new update is added.\n>>\n>> - Modify the backends to use this field internally as needed. The\n>> backends need to check if an update for refname already exists when\n>> splitting symrefs or adding an update for 'HEAD'.\n>\n> Okay. Is this actually necessary to be handled by the backends? I\n> would've expected that it is possible to split up symref updates so that\n> we insert both symref and target into the list. I wouldn't be surprised\n> if this wasn't easily possible though -- the logic here is surprisingly\n> intricate.\n\nIt is a bit intricate and requires a bit of unwinding to move it to the\ngeneric layer. But it is possible, I tried to timebox it for this patch\nseries, but unfortunately it needs a lot more time. So perhaps,\nsomething for later.\n\n>\n>> - In the reftable backend, in `reftable_be_transaction_prepare()`,\n>> move the instance of `string_list_has_string()` above\n>> `ref_transaction_add_update()` to check before the reference is\n>> added.\n>>\n>> This helps reduce duplication of functionality between the backends and\n>> makes it easier to make changes in a more centralized manner.\n>\n>> Signed-off-by: Karthik Nayak \n>> ---\n>> refs.c | 17 ++++++++++++\n>> refs/files-backend.c | 69 ++++++++++---------------------------------------\n>> refs/packed-backend.c | 25 +-----------------\n>> refs/refs-internal.h | 2 ++\n>> refs/reftable-backend.c | 53 ++++++++++++-------------------------\n>> 5 files changed, 50 insertions(+), 116 deletions(-)\n>\n> Nice.\n>\n>> diff --git a/refs.c b/refs.c\n>> index f4094a326a9f88f979654b668cc9c3d27d83cb5d..4c9b706461977995be1d55e7667f7fb708fbbb76 100644\n>> --- a/refs.c\n>> +++ b/refs.c\n>> @@ -1175,6 +1175,7 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,\n>> \tCALLOC_ARRAY(tr, 1);\n>> \ttr->ref_store = refs;\n>> \ttr->flags = flags;\n>> +\tstring_list_init_dup(&tr->refnames);\n>\n> Do we actually have to duplicate strings? I would've expected that we\n> keep strings alive via the `ref_update`s anyway during the transaction's\n> lifetime.\n>\n\nTrue. I was more thinking along the lines of the keeping the memory\nconcerns separate. Also, I sure if there are any scenario's that\na `ref_transaction` could outlive a `ref_update`.\n\n> It might also be interesting to check whether using a strset for this\n> is more efficient. But that is certainly outside the scope of your patch\n> series and can be done at a later point. #leftoverbit\n>\n\nAgreed.\n\n>> @@ -1245,6 +1248,16 @@ struct ref_update *ref_transaction_add_update(\n>> \t\tupdate->msg = normalize_reflog_message(msg);\n>> \t}\n>>\n>> +\t/*\n>> +\t * This list is generally used by the backends to avoid duplicates.\n>> +\t * But we do support multiple log updates for a given refname within\n>> +\t * a single transaction.\n>> +\t */\n>> +\tif (!(update->flags & REF_LOG_ONLY)) {\n>> +\t\titem = string_list_append(&transaction->refnames, refname);\n>> +\t\titem->util = update;\n>> +\t}\n>> +\n>> \treturn update;\n>> }\n>> @@ -2397,6 +2410,10 @@ int ref_transaction_prepare(struct ref_transaction *transaction,\n>> \t\treturn -1;\n>> \t}\n>>\n>> +\tstring_list_sort(&transaction->refnames);\n>> +\tif (ref_update_reject_duplicates(&transaction->refnames, err))\n>> +\t\treturn TRANSACTION_GENERIC_ERROR;\n>> +\n>> \tret = refs->be->transaction_prepare(refs, transaction, err);\n>> \tif (ret)\n>> \t\treturn ret;\n>\n> Okay, we keep the list unserted initially, but sort it later before\n> passing it to the backends so that `string_list_has_string()` works\n> correctly. Good.\n>\n>> diff --git a/refs/files-backend.c b/refs/files-backend.c\n>> index c6a3f6d6261a894e1c294bb1329fdf8079a39eb4..18da30c3f37dc5c09f7d81a9083d6b41d0463bd5 100644\n>> --- a/refs/files-backend.c\n>> +++ b/refs/files-backend.c\n>> @@ -2425,7 +2423,6 @@ static int split_head_update(struct ref_update *update,\n>> \t */\n>> \tif (strcmp(new_update->refname, \"HEAD\"))\n>> \t\tBUG(\"%s unexpectedly not 'HEAD'\", new_update->refname);\n>> -\tstring_list_insert(affected_refnames, new_update->refname);\n>>\n>> \treturn 0;\n>> }\n>\n> Previously we would've inserted \"HEAD\" into the list of affected\n> refnames even if it wasn't directly updated. Why don't we have to do\n> that now anymore?\n>\n\nWe still do, right above this code, there is a call to\n`ref_transaction_add_update()`. So any ref_update added to the\ntransaction via `ref_transaction_add_update()` will also add the refname\nto `transaction->refnames`.\n\n>> @@ -2441,7 +2438,6 @@ static int split_head_update(struct ref_update *update,\n>> @@ -2491,15 +2487,6 @@ static int split_symref_update(struct ref_update *update,\n>> \tupdate->flags |= REF_LOG_ONLY | REF_NO_DEREF;\n>> \tupdate->flags &= ~REF_HAVE_OLD;\n>>\n>> -\t/*\n>> -\t * Add the referent. This insertion is O(N) in the transaction\n>> -\t * size, but it happens at most once per symref in a\n>> -\t * transaction. Make sure to add new_update->refname, which will\n>> -\t * be valid as long as affected_refnames is in use, and NOT\n>> -\t * referent, which might soon be freed by our caller.\n>> -\t */\n>> -\tstring_list_insert(affected_refnames, new_update->refname);\n>> -\n>> \treturn 0;\n>> }\n>\n> Same question here, but for symref updates.\n>\n\nSame as above. In summary, the need for adding new refnames to the list\nis now centralized and a part of adding a ref update to the transaction.\n\nIt's a good question, so I'll also add a hint in the commit message.\n\n>\n>> @@ -3030,13 +2995,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n>> \tif (transaction->state != REF_TRANSACTION_PREPARED)\n>> \t\tBUG(\"commit called for transaction that is not prepared\");\n>>\n>> -\t/* Fail if a refname appears more than once in the transaction: */\n>> -\tfor (i = 0; i < transaction->nr; i++)\n>> -\t\tif (!(transaction->updates[i]->flags & REF_LOG_ONLY))\n>> -\t\t\tstring_list_append(&affected_refnames,\n>> -\t\t\t\t\t transaction->updates[i]->refname);\n>> -\tstring_list_sort(&affected_refnames);\n>> -\tif (ref_update_reject_duplicates(&affected_refnames, err)) {\n>> +\tstring_list_sort(&transaction->refnames);\n>> +\tif (ref_update_reject_duplicates(&transaction->refnames, err)) {\n>> \t\tret = TRANSACTION_GENERIC_ERROR;\n>> \t\tgoto cleanup;\n>> \t}\n>\n> Can't we also make this check generic for initial transactions?\n>\n\nThis one is handled in the next commit, I mostly separated them out\nbecause I was not sure why this needs to be here and to draw attention\nif I'm missing something when removing this.\n\n>> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n>> index a7b6f74b6e35f897f619c540cbc600bbd888bc67..6e7acb077e81435715a1ca3cc928550147c8c56a 100644\n>> --- a/refs/packed-backend.c\n>> +++ b/refs/packed-backend.c\n>> @@ -1653,34 +1648,16 @@ static int packed_transaction_prepare(struct ref_store *ref_store,\n>> \t */\n>>\n>> \tCALLOC_ARRAY(data, 1);\n>> -\tstring_list_init_nodup(&data->updates);\n>>\n>> \ttransaction->backend_data = data;\n>>\n>> -\t/*\n>> -\t * Stick the updates in a string list by refname so that we\n>> -\t * can sort them:\n>> -\t */\n>> -\tfor (i = 0; i < transaction->nr; i++) {\n>> -\t\tstruct ref_update *update = transaction->updates[i];\n>> -\t\tstruct string_list_item *item =\n>> -\t\t\tstring_list_append(&data->updates, update->refname);\n>> -\n>> -\t\t/* Store a pointer to update in item->util: */\n>> -\t\titem->util = update;\n>> -\t}\n>> -\tstring_list_sort(&data->updates);\n>> -\n>> -\tif (ref_update_reject_duplicates(&data->updates, err))\n>> -\t\tgoto failure;\n>> -\n>> \tif (!is_lock_file_locked(&refs->lock)) {\n>> \t\tif (packed_refs_lock(ref_store, 0, err))\n>> \t\t\tgoto failure;\n>> \t\tdata->own_lock = 1;\n>> \t}\n>>\n>> -\tif (write_with_updates(refs, &data->updates, err))\n>> +\tif (write_with_updates(refs, &transaction->refnames, err))\n>> \t\tgoto failure;\n>>\n>> \ttransaction->state = REF_TRANSACTION_PREPARED;\n>\n> This change is a lot more straight-forward because the packed backend\n> does not support symrefs at all. Nice.\n>\n\nYes indeed.\n\n>> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\n>> index d39a14c5a469d7d219362e9eae4f578784d65a5b..dd2099d94948a4f23fd9f7ddc06bf3d741229eba 100644\n>> --- a/refs/reftable-backend.c\n>> +++ b/refs/reftable-backend.c\n>> @@ -1202,12 +1184,11 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n>> \t\t\t\tgoto done;\n>> \t\t\t}\n>>\n>> -\t\t\tnew_update = ref_transaction_add_update(\n>> -\t\t\t\t\ttransaction, \"HEAD\",\n>> -\t\t\t\t\tu->flags | REF_LOG_ONLY | REF_NO_DEREF,\n>> -\t\t\t\t\t&u->new_oid, &u->old_oid, NULL, NULL, NULL,\n>> -\t\t\t\t\tu->msg);\n>> -\t\t\tstring_list_insert(&affected_refnames, new_update->refname);\n>> +\t\t\tref_transaction_add_update(\n>> +\t\t\t\ttransaction, \"HEAD\",\n>> +\t\t\t\tu->flags | REF_LOG_ONLY | REF_NO_DEREF,\n>> +\t\t\t\t&u->new_oid, &u->old_oid, NULL, NULL, NULL,\n>> +\t\t\t\tu->msg);\n>> \t\t}\n>>\n>> \t\tret = reftable_backend_read_ref(be, rewritten_ref,\n>\n> Equivalent question as for the files backend.\n>\n\nI hope my answer earlier helps, especially since the diff here shows the\ncall to `ref_transaction_add_update()`.\n\n>> @@ -1277,6 +1258,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n>> \t\t\t\tif (!strcmp(rewritten_ref, \"HEAD\"))\n>> \t\t\t\t\tnew_flags |= REF_UPDATE_VIA_HEAD;\n>>\n>> +\t\t\t\tif (string_list_has_string(&transaction->refnames, referent.buf)) {\n>> +\t\t\t\t\tstrbuf_addf(err,\n>> +\t\t\t\t\t\t _(\"multiple updates for '%s' (including one \"\n>> +\t\t\t\t\t\t \"via symref '%s') are not allowed\"),\n>> +\t\t\t\t\t\t referent.buf, u->refname);\n>> +\t\t\t\t\tret = TRANSACTION_NAME_CONFLICT;\n>> +\t\t\t\t\tgoto done;\n>> +\t\t\t\t}\n>> +\n>> \t\t\t\t/*\n>> \t\t\t\t * If we are updating a symref (eg. HEAD), we should also\n>> \t\t\t\t * update the branch that the symref points to.\n>\n> This change surprised me a bit. You mention it in the commit message,\n> but don't state a reason why you do it.\n>\n\nWhen a `ref_update` is added to the `transaction` via\n`ref_transaction_add_update()`, the refname is automatically added to\n`transaction->refnames`. As a result, checking for the refname in this\nlist will always return true. I'll clarify this in the commit message.\n\n> Patrick\n"},{"id":"512240","messageId":"4beb0359-763d-425d-b416-ac40bda59e2e@gmail.com","threadId":"62915","inReplyTo":"20250207-245-partially-atomic-ref-updates-v1-0-e6a3690ff23a@gmail.com","subject":"Re: [PATCH 0/6] refs: introduce support for partial reference transactions","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2025-02-11T17:03:56Z","receivedAt":"2025-02-11T17:03:59Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Karthik\n\nOn 07/02/2025 07:34, Karthik Nayak wrote:\n> Git's reference updates are traditionally atomic\n\nI'm nitpicking but the updates aren't actually atomic, if a transaction \nupdates two refs then it is possible for another process to see the one \nref pointing to the new value and the other pointing to the old value.\n\n> - when updating\n> multiple references in a transaction, either all updates succeed or none\n> do. While this behavior is generally desirable,\n\nIsn't that the whole point of transactions?\n\n> it can be limiting in> certain scenarios, particularly with the reftable backend where batching\n> multiple reference updates is more efficient than performing them\n> sequentially.\n> \n> This series introduces support for partial reference transactions,\n> allowing individual reference updates to fail while letting others\n> proceed.\n\nThis sounds like it's abusing ref transactions to implement a \nperformance optimization. I wonder if it would be better to provide that \nvia a different interface than shares the same underling implementation \nas transactions. That would make it clear to someone reading the code \nthat individual ref updates can fail without affecting the rest. Burying \nthat detail in a flag makes it rather easy to miss.\n\nBest Wishes\n\nPhillip\n\n> This capability is exposed through git-update-ref's\n> `--allow-partial` flag, which can be used in `--stdin` mode to batch\n> updates and handle failures gracefully.\n> The changes are structured to carefully build up this functionality:\n> \n> First, we clean up and consolidate the reference update checking logic.\n> This includes removing duplicate checks in the files backend and moving\n> refname tracking to the generic layer, which simplifies the codebase and\n> prepares it for the new feature.\n> \n> We then restructure the reftable backend's transaction preparation code,\n> extracting the update validation logic into a dedicated function. This\n> not only improves code organization but sets the stage for implementing\n> partial transaction support.\n> \n> With this groundwork in place, we implement the core partial transaction\n> support in the refs subsystem. This adds the necessary infrastructure to\n> track and report rejected updates while allowing transactions to proceed.\n> All reference backends are modified to support this behavior when enabled.\n> \n> Finally, we expose this functionality to users through\n> git-update-ref(1)'s `--allow-partial` flag, complete with test coverage\n> and documentation. The flag is specifically limited to `--stdin` mode\n> where batching multiple updates is most relevant.\n> \n> This enhancement improves Git's flexibility in handling reference\n> updates while maintaining the safety of atomic transactions by default.\n> It's particularly valuable for tools and workflows that need to handle\n> reference update failures gracefully without abandoning the entire batch\n> of updates.\n> \n> This series is based on top of bc204b7427 (The seventh batch, 2025-02-03).\n> There were no conflicts noticed with topics in 'seen' or 'next'.\n> \n> ---\n> Karthik Nayak (6):\n> refs/files: remove duplicate check in `split_symref_update()`\n> refs: move duplicate refname update check to generic layer\n> refs/files: remove duplicate duplicates check\n> refs/reftable: extract code from the transaction preparation\n> refs: implement partial reference transaction support\n> update-ref: add --allow-partial flag for stdin mode\n> \n> Documentation/git-update-ref.txt | 12 +-\n> builtin/update-ref.c | 53 ++++-\n> refs.c | 58 ++++-\n> refs.h | 22 ++\n> refs/files-backend.c | 97 ++------\n> refs/packed-backend.c | 49 ++--\n> refs/refs-internal.h | 19 +-\n> refs/reftable-backend.c | 494 +++++++++++++++++++--------------------\n> t/t1400-update-ref.sh | 191 +++++++++++++++\n> 9 files changed, 633 insertions(+), 362 deletions(-)\n> ---\n> \n> \n> \n> ---\n> \n> base-commit: bc204b742735ae06f65bb20291c95985c9633b7f\n> change-id: 20241206-245-partially-atomic-ref-updates-9fe8b080345c\n> \n> Thanks\n> - Karthik\n> \n> \n\n"},{"id":"512243","messageId":"786eef7b-94e5-4f92-a82c-aeea69b5c103@gmail.com","threadId":"62915","inReplyTo":"4beb0359-763d-425d-b416-ac40bda59e2e@gmail.com","subject":"Re: [PATCH 0/6] refs: introduce support for partial reference transactions","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2025-02-11T17:40:35Z","receivedAt":"2025-02-11T17:40:38Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 11/02/2025 17:03, Phillip Wood wrote:\n> On 07/02/2025 07:34, Karthik Nayak wrote:\n >\n>> This series introduces support for partial reference transactions,\n>> allowing individual reference updates to fail while letting others\n>> proceed.\n\nThinking about this some more it is possible to skip the checking the \ncurrent value of the ref so what is making the transaction fail? Is it \nD/F conflicts or something else?\n\nBest Wishes\n\nPhillip\n\n"},{"id":"512313","messageId":"CAOLa=ZQF0=WgW-2HumRbSp4rUjXikEfecjp5Uxp+zJ+Tun5yzw@mail.gmail.com","threadId":"62915","inReplyTo":"4beb0359-763d-425d-b416-ac40bda59e2e@gmail.com","subject":"Re: [PATCH 0/6] refs: introduce support for partial reference transactions","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-02-12T12:34:21Z","receivedAt":"2025-02-12T12:34:23Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Phillip Wood writes:\n\n> Hi Karthik\n>\n> On 07/02/2025 07:34, Karthik Nayak wrote:\n>> Git's reference updates are traditionally atomic\n>\n> I'm nitpicking but the updates aren't actually atomic, if a transaction\n> updates two refs then it is possible for another process to see the one\n> ref pointing to the new value and the other pointing to the old value.\n>\n\nGood point. This is true in the case of the files backend, since updates\ninvolve locking individual files and during the commit phase, there is a\npossibility that one ref is updated while the other is yet to be\n(committing of the lock is not global but rather per ref file).\n\nHowever this is not the case with the reftable backend, there, updates\nare written to a new table and committed at the end after locking the\ntable. So in the reftable backend, this is indeed atomic.\n\n>> - when updating\n>> multiple references in a transaction, either all updates succeed or none\n>> do. While this behavior is generally desirable,\n>\n> Isn't that the whole point of transactions?\n>\n\nYup, this is the point of having a transaction indeed.\n\n>> it can be limiting in> certain scenarios, particularly with the reftable backend where batching\n>> multiple reference updates is more efficient than performing them\n>> sequentially.\n>>\n>> This series introduces support for partial reference transactions,\n>> allowing individual reference updates to fail while letting others\n>> proceed.\n>\n> This sounds like it's abusing ref transactions to implement a\n> performance optimization.\n\nI understand where you're coming from. This is definitely a stray from\nthe regular atomic behavior, that transactions promise. But I would say\nthis is more of an exception handling for the regular transaction\nmechanism and AFAIK this is also something that some of the databases\nsupport (see EXCEPTION in PostgreSQL).\n\nOverall, we're adding an exception handling support to the existing\ntransaction interface.\n\n> I wonder if it would be better to provide that\n> via a different interface than shares the same underling implementation\n> as transactions. That would make it clear to someone reading the code\n> that individual ref updates can fail without affecting the rest. Burying\n> that detail in a flag makes it rather easy to miss.\n>\n\nThinking this out, having a different interface sound good, but I feel\nwe'd end up with the same structure as currently presented in this\nseries. Only other way is to really split the implementation to support\npartial transactions as a entity of its own. In that case, we'd end up\nwith code duplication.\n\nDo you think you can expand a little more here?\n\n> Best Wishes\n>\n> Phillip\n>\n\nThanks for engaging!\n\n[snip]\n"},{"id":"512314","messageId":"CAOLa=ZREofXJnyFEzQLoAKk-A1_==wqBy3YyCCgLeXT8qhdGHg@mail.gmail.com","threadId":"62915","inReplyTo":"786eef7b-94e5-4f92-a82c-aeea69b5c103@gmail.com","subject":"Re: [PATCH 0/6] refs: introduce support for partial reference transactions","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-02-12T12:36:56Z","receivedAt":"2025-02-12T12:36:58Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Phillip Wood writes:\n\n> On 11/02/2025 17:03, Phillip Wood wrote:\n>> On 07/02/2025 07:34, Karthik Nayak wrote:\n> >\n>>> This series introduces support for partial reference transactions,\n>>> allowing individual reference updates to fail while letting others\n>>> proceed.\n>\n> Thinking about this some more it is possible to skip the checking the\n> current value of the ref so what is making the transaction fail? Is it\n> D/F conflicts or something else?\n>\n\nIt could be a multitude of issues, to name a some:\n - D/F conflicts\n - Unexpected old oid/target\n - dangling symrefs\n\nSo this gives a hatch to partially commit parts of a transaction while\nalso notifying the user about parts which failed.\n\n> Best Wishes\n>\n> Phillip\n"},{"id":"512710","messageId":"282b132e-27b0-41b7-8556-cbbdc08081bc@gmail.com","threadId":"62915","inReplyTo":"CAOLa=ZQF0=WgW-2HumRbSp4rUjXikEfecjp5Uxp+zJ+Tun5yzw@mail.gmail.com","subject":"Re: [PATCH 0/6] refs: introduce support for partial reference transactions","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2025-02-19T14:34:15Z","receivedAt":"2025-02-19T14:34:18Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Karthik\n\nOn 12/02/2025 12:34, Karthik Nayak wrote:\n>> On 07/02/2025 07:34, Karthik Nayak wrote:\n>>> Git's reference updates are traditionally atomic\n>>\n>> I'm nitpicking but the updates aren't actually atomic, if a transaction\n>> updates two refs then it is possible for another process to see the one\n>> ref pointing to the new value and the other pointing to the old value.\n>>\n> \n> Good point. This is true in the case of the files backend, since updates\n> involve locking individual files and during the commit phase, there is a\n> possibility that one ref is updated while the other is yet to be\n> (committing of the lock is not global but rather per ref file).\n> \n> However this is not the case with the reftable backend, there, updates\n> are written to a new table and committed at the end after locking the\n> table. So in the reftable backend, this is indeed atomic.\n\nAh, interesting. That explains why batching updates is so much more \nefficient when using the reftable backend.\n\n>>> This series introduces support for partial reference transactions,\n>>> allowing individual reference updates to fail while letting others\n>>> proceed.\n>>\n>> This sounds like it's abusing ref transactions to implement a\n>> performance optimization.\n> \n> I understand where you're coming from. This is definitely a stray from\n> the regular atomic behavior, that transactions promise. But I would say\n> this is more of an exception handling for the regular transaction\n> mechanism and AFAIK this is also something that some of the databases\n> support (see EXCEPTION in PostgreSQL).\n> \n> Overall, we're adding an exception handling support to the existing\n> transaction interface.\n\nMy understanding of exception handling is that if an error occurs then \nan error handler is called (reading [1] that seems to be what PostgreSQL \ndoes as well). Is that what is being proposed here? I thought this \nseries added a flag to ignore errors rather than provide a way to handle \nthem.\n\n[1] \nhttps://www.postgresql.org/docs/current/plpgsql-control-structures.html#PLPGSQL-ERROR-TRAPPING\n\n>> I wonder if it would be better to provide that\n>> via a different interface than shares the same underling implementation\n>> as transactions. That would make it clear to someone reading the code\n>> that individual ref updates can fail without affecting the rest. Burying\n>> that detail in a flag makes it rather easy to miss.\n>>\n> \n> Thinking this out, having a different interface sound good, but I feel\n> we'd end up with the same structure as currently presented in this\n> series. Only other way is to really split the implementation to support\n> partial transactions as a entity of its own. In that case, we'd end up\n> with code duplication.\n> \n> Do you think you can expand a little more here?\n\nI was thinking of a function that took a list of refs to update and made \na best effort to update them, ignoring any updates that fail.\n\nMy concern with adding a flag to ignore errors in the transaction api is \nthat a partial transaction is a contradiction in terms. I'm also \nconcerned that it seems to be ignoring all errors. I'd be happier if \nthere was someway for the caller to specify which errors to ignore or if \nthe caller could provide a callback to handle any errors. That way a \ncaller could ignore d/f conflicts but still cause the transaction to \nfail if there was an i/o or could create a reference if it did not exist \nbut leave it unchanged if it did exist.\n\nBest Wishes\n\nPhillip\n\n"},{"id":"512714","messageId":"Z7X0ga1nmRheAQJ4@pks.im","threadId":"62915","inReplyTo":"282b132e-27b0-41b7-8556-cbbdc08081bc@gmail.com","subject":"Re: [PATCH 0/6] refs: introduce support for partial reference transactions","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-19T15:10:57Z","receivedAt":"2025-02-19T15:11:04Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Feb 19, 2025 at 02:34:15PM +0000, Phillip Wood wrote:\n> On 12/02/2025 12:34, Karthik Nayak wrote:\n> > Thinking this out, having a different interface sound good, but I feel\n> > we'd end up with the same structure as currently presented in this\n> > series. Only other way is to really split the implementation to support\n> > partial transactions as a entity of its own. In that case, we'd end up\n> > with code duplication.\n> > \n> > Do you think you can expand a little more here?\n> \n> I was thinking of a function that took a list of refs to update and made a\n> best effort to update them, ignoring any updates that fail.\n> \n> My concern with adding a flag to ignore errors in the transaction api is\n> that a partial transaction is a contradiction in terms. I'm also concerned\n> that it seems to be ignoring all errors. I'd be happier if there was someway\n> for the caller to specify which errors to ignore or if the caller could\n> provide a callback to handle any errors. That way a caller could ignore d/f\n> conflicts but still cause the transaction to fail if there was an i/o or\n> could create a reference if it did not exist but leave it unchanged if it\n> did exist.\n\nThis is a fair point I think, and it's also something that I called out\nin [1]. We shouldn't blanket-ignore all errors, but instead only ignore\na well-known subset of errors and then record the exact failure reason.\nThis allows for better and more unified error reporting, and would also\nallow us to easily build mechanisms where callers can specify that we\nare only expected to ignore a subset of those well-defined errors.\n\nBut I also think that this is another selling point for continuing to\nbuild on top of the ref transaction. If we now want to record and ignore\nspecific errors, only, then this falls out quite naturally from the\ncurrent design of ref transactions. We already have all the ref updates\nin there, so we can then simply set an `enum ref_transaction_error`\nfield for each of the failed updates.\n\nEventually, I think we could also allow for modes where we declare only\na subset of reference updates to be allowed to fail. I don't have a\nspecific usecase for this, and don't think it needs to be implemented\nwithout one. But it's another thing that we could implement on top of\ntransactions rather trivially.\n\nPatrick\n\n[1]: \n"},{"id":"512820","messageId":"CAOLa=ZQG7UOiYf94jXJwm5SC69hwXCpv2HRCXxMCOUONnzKaUA@mail.gmail.com","threadId":"62915","inReplyTo":"Z6YxA4BlhNwbeYk-@pks.im","subject":"Re: [PATCH 5/6] refs: implement partial reference transaction support","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-02-21T10:33:38Z","receivedAt":"2025-02-21T10:33:39Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> On Fri, Feb 07, 2025 at 08:34:40AM +0100, Karthik Nayak wrote:\n>> Git's reference transactions are all-or-nothing: either all updates\n>> succeed, or none do. While this atomic behavior is generally desirable,\n>> it can be suboptimal when using the reftable backend, where batching\n>> multiple reference updates into a single transaction is more efficient\n>> than performing them sequentially.\n>\n> In fact it's even inefficient for the \"files\" backend. The whole\n> machinery around creating a new transaction, preparing it, committing it\n> and then cleaning up its state does bring a bunch of overhead with it.\n> But true, for the \"reftable\" backend it's way more impactful.\n>\n>> diff --git a/refs.c b/refs.c\n>> index b420a120102b3793168598b885bba68e4f5f5f03..75dbd84acbc41658d4b8b6b5e7763c04e78d0061 100644\n>> --- a/refs.c\n>> +++ b/refs.c\n>> @@ -1211,6 +1212,14 @@ void ref_transaction_free(struct ref_transaction *transaction)\n>> \tfree(transaction);\n>> }\n>>\n>> +void ref_transaction_add_rejection(struct ref_transaction *transaction,\n>> +\t\t\t\t size_t update_idx, struct strbuf *err)\n>\n> \"add\" to me sounds like you're adding a new thingy to the transaction,\n> but you rather update something. How about `ref_update_set_rejected()`\n> or `ref_transacton_set_rejected()`?\n>\n\nFair enough, I've changed it to `ref_transacton_set_rejected()`.\n\n>> +{\n>> +\tstruct ref_update *update = transaction->updates[update_idx];\n>\n> Do we want to `BUG()` in case `update_idx >= transaction->nr`?\n>\n\nGood point, let me add that in.\n\n>> +\tupdate->rejected = 1;\n>> +\tstrbuf_addbuf(&update->rejection_err, err);\n>> +}\n>\n> Do we really need a string as rejection error? I'd expect that the set\n> of failures that lead to rejection should be rather limited, which means\n> that we could use an enum instead. This would unify the errors across\n> backends and also allows us to figure out the root cause of rejection in\n> other subsystems.\n>\n> If we introduced an enum, we could eventually even iterate a bit on the\n> mechanism and rather trivially tell the backends which kind of failures\n> are acceptable. As an example, a conflicting ref update may for example\n> be ignored and not cause failure, a conflicting path name might cause\n> failure.\n>\n\nThat's a good point, This also allows us to eventually extend the flag\nto do something like you mentioned where `--allow-partial=all` would\nskip all errors. But one could optimize to also say\n`--allow-partial=name_conflict,old_value` to only skip errors due to\nrefname conflicts and invalid/incorrect old_value.\n\nI'll add another commit to introduce and add 'enum transaction_error'\nand build around it.\n\n>> diff --git a/refs/files-backend.c b/refs/files-backend.c\n>> index 9fc5454678340dd7c72539bfa0f15ee7eb24b1ff..99ec29164fbd30635125cc2325aab3d300cf906c 100644\n>> --- a/refs/files-backend.c\n>> +++ b/refs/files-backend.c\n>> @@ -2852,8 +2852,18 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n>>\n>> \t\tret = lock_ref_for_update(refs, update, transaction,\n>> \t\t\t\t\t head_ref, err);\n>> -\t\tif (ret)\n>> +\t\tif (ret) {\n>\n> I wonder whether we want to accept all failures. Some failures are\n> certainly benign, like for example mismatching expected OIDs or a\n> conflict due to a preexisting ref that blocks the path. But other kinds\n> of failures which are unexpected might be a bit more on the dangerous\n> side to accept, so I think we should be careful here.\n>\n\nFair point. For the current implementation with the enum design\ndiscussed above. I think it would be best to skip over all user\noriented errors. But any system errors would actually cease the\ntransaction. We can further iterate on this later.\n\n> The same comment also applies to the other backends.\n>\n> Patrick\n"},{"id":"512821","messageId":"CAOLa=ZT_PVwD0=hFEEqxutFyrQRiz3f4iB8019Sax7_nEwrJfw@mail.gmail.com","threadId":"62915","inReplyTo":"Z6YxB6qL8WQFZyWW@pks.im","subject":"Re: [PATCH 6/6] update-ref: add --allow-partial flag for stdin mode","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-02-21T11:45:42Z","receivedAt":"2025-02-21T11:45:44Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> On Fri, Feb 07, 2025 at 08:34:41AM +0100, Karthik Nayak wrote:\n>> diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt\n>> index 9e6935d38d031b4890135e0cce36fffcc349ac1d..529d3c15404cdc13216219fba6f56dde91f4909c 100644\n>> --- a/Documentation/git-update-ref.txt\n>> +++ b/Documentation/git-update-ref.txt\n>> @@ -8,7 +8,7 @@ git-update-ref - Update the object name stored in a ref safely\n>> SYNOPSIS\n>> --------\n>> [verse]\n>> -'git update-ref' [-m ] [--no-deref] (-d [] | [--create-reflog] [] | --stdin [-z])\n>> +'git update-ref' [-m ] [--no-deref] (-d [] | [--create-reflog] [] | --stdin [-z] [--allow-partial])\n>\n> I think it's time that we start to split this line into multiple lines :)\n>\n\nYes, indeed, will do.\n\n>> diff --git a/builtin/update-ref.c b/builtin/update-ref.c\n>> index 4d35bdc4b4b57937112e6c4c9740420b1f1771e5..83dcb7d8d73f423226c36b61374c86c6b29ec756 100644\n>> --- a/builtin/update-ref.c\n>> +++ b/builtin/update-ref.c\n>> @@ -562,6 +563,30 @@ static void parse_cmd_abort(struct ref_transaction *transaction,\n>> \treport_ok(\"abort\");\n>> }\n>>\n>> +static void print_rejected_refs(const char *refname,\n>> +\t\t\t\tconst struct object_id *old_oid,\n>> +\t\t\t\tconst struct object_id *new_oid,\n>> +\t\t\t\tconst char *old_target,\n>> +\t\t\t\tconst char *new_target,\n>> +\t\t\t\tconst struct strbuf *reason,\n>> +\t\t\t\tvoid *cb_data UNUSED)\n>> +{\n>> +\tstruct strbuf sb = STRBUF_INIT;\n>> +\tchar space = ' ';\n>> +\n>> +\tif (!line_termination)\n>> +\t\tspace = line_termination;\n>> +\n>> +\tstrbuf_addf(&sb, \"rejected%c%s%c%s%c%c%s%c%s%c\", space,\n>\n> Whew, that's a lot of placeholders.\n>\n\nTrue. More prone to errors too.\n\n>> @@ -723,7 +754,8 @@ int cmd_update_ref(int argc,\n>> \tconst char *refname, *oldval;\n>> \tstruct object_id oid, oldoid;\n>> \tint delete = 0, no_deref = 0, read_stdin = 0, end_null = 0;\n>> -\tint create_reflog = 0;\n>> +\tint create_reflog = 0, allow_partial = 0;\n>> +\n>> \tstruct option options[] = {\n>> \t\tOPT_STRING( 'm', NULL, &msg, N_(\"reason\"), N_(\"reason of the update\")),\n>> \t\tOPT_BOOL('d', NULL, &delete, N_(\"delete the reference\")),\n>> @@ -732,6 +764,7 @@ int cmd_update_ref(int argc,\n>> \t\tOPT_BOOL('z', NULL, &end_null, N_(\"stdin has NUL-terminated arguments\")),\n>> \t\tOPT_BOOL( 0 , \"stdin\", &read_stdin, N_(\"read updates from stdin\")),\n>> \t\tOPT_BOOL( 0 , \"create-reflog\", &create_reflog, N_(\"create a reflog\")),\n>> +\t\tOPT_BOOL('0', \"allow-partial\", &allow_partial, N_(\"allow partial transactions\")),\n>\n> You can use `OPT_BIT()` to set a specific bit in a flags field..\n>\n\nThat would be cleaner, will fix.\n\n>> @@ -749,13 +782,19 @@ int cmd_update_ref(int argc,\n>> \t}\n>>\n>> \tif (read_stdin) {\n>> +\t\tunsigned int flags = 0;\n>> +\n>> +\t\tif (allow_partial)\n>> +\t\t\tflags |= REF_TRANSACTION_ALLOW_PARTIAL;\n>> +\n>> \t\tif (delete || argc > 0)\n>> \t\t\tusage_with_options(git_update_ref_usage, options);\n>> \t\tif (end_null)\n>> \t\t\tline_termination = '\\0';\n>> -\t\tupdate_refs_stdin();\n>> +\t\tupdate_refs_stdin(flags);\n>> \t\treturn 0;\n>> -\t}\n>> +\t} else if (allow_partial)\n>> +\t\tdie(\"--allow-partial can only be used with --stdin\");\n>>\n>> \tif (end_null)\n>> \t\tusage_with_options(git_update_ref_usage, options);\n>\n> The implementation is quite simple, nice.\n>\n> Patrick\n"},{"id":"512822","messageId":"CAOLa=ZQHMsZK0rmb2MuJYuaR-+1s6CdEQm86caKc_qXMSOKPTw@mail.gmail.com","threadId":"62915","inReplyTo":"282b132e-27b0-41b7-8556-cbbdc08081bc@gmail.com","subject":"Re: [PATCH 0/6] refs: introduce support for partial reference transactions","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-02-21T11:50:23Z","receivedAt":"2025-02-21T11:50:26Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Phillip Wood writes:\n\n> Hi Karthik\n>\n> On 12/02/2025 12:34, Karthik Nayak wrote:\n>>> On 07/02/2025 07:34, Karthik Nayak wrote:\n>>>> Git's reference updates are traditionally atomic\n>>>\n>>> I'm nitpicking but the updates aren't actually atomic, if a transaction\n>>> updates two refs then it is possible for another process to see the one\n>>> ref pointing to the new value and the other pointing to the old value.\n>>>\n>>\n>> Good point. This is true in the case of the files backend, since updates\n>> involve locking individual files and during the commit phase, there is a\n>> possibility that one ref is updated while the other is yet to be\n>> (committing of the lock is not global but rather per ref file).\n>>\n>> However this is not the case with the reftable backend, there, updates\n>> are written to a new table and committed at the end after locking the\n>> table. So in the reftable backend, this is indeed atomic.\n>\n> Ah, interesting. That explains why batching updates is so much more\n> efficient when using the reftable backend.\n>\n>>>> This series introduces support for partial reference transactions,\n>>>> allowing individual reference updates to fail while letting others\n>>>> proceed.\n>>>\n>>> This sounds like it's abusing ref transactions to implement a\n>>> performance optimization.\n>>\n>> I understand where you're coming from. This is definitely a stray from\n>> the regular atomic behavior, that transactions promise. But I would say\n>> this is more of an exception handling for the regular transaction\n>> mechanism and AFAIK this is also something that some of the databases\n>> support (see EXCEPTION in PostgreSQL).\n>>\n>> Overall, we're adding an exception handling support to the existing\n>> transaction interface.\n>\n> My understanding of exception handling is that if an error occurs then\n> an error handler is called (reading [1] that seems to be what PostgreSQL\n> does as well). Is that what is being proposed here? I thought this\n> series added a flag to ignore errors rather than provide a way to handle\n> them.\n>\n\nThat is correct, and while the current implementation is to ignore them.\nIt does make way for building something like that in the future.\n\n> [1]\n> https://www.postgresql.org/docs/current/plpgsql-control-structures.html#PLPGSQL-ERROR-TRAPPING\n>\n>>> I wonder if it would be better to provide that\n>>> via a different interface than shares the same underling implementation\n>>> as transactions. That would make it clear to someone reading the code\n>>> that individual ref updates can fail without affecting the rest. Burying\n>>> that detail in a flag makes it rather easy to miss.\n>>>\n>>\n>> Thinking this out, having a different interface sound good, but I feel\n>> we'd end up with the same structure as currently presented in this\n>> series. Only other way is to really split the implementation to support\n>> partial transactions as a entity of its own. In that case, we'd end up\n>> with code duplication.\n>>\n>> Do you think you can expand a little more here?\n>\n> I was thinking of a function that took a list of refs to update and made\n> a best effort to update them, ignoring any updates that fail.\n>\n\nYes, this is what we want too, but in the context of transactions.\nWithout transactions, this is already possible to build albeit on the\nuser side with some simple scripts.\n\n> My concern with adding a flag to ignore errors in the transaction api is\n> that a partial transaction is a contradiction in terms. I'm also\n> concerned that it seems to be ignoring all errors. I'd be happier if\n> there was someway for the caller to specify which errors to ignore or if\n> the caller could provide a callback to handle any errors. That way a\n> caller could ignore d/f conflicts but still cause the transaction to\n> fail if there was an i/o or could create a reference if it did not exist\n> but leave it unchanged if it did exist.\n>\n\nYeah, I think this is also something that Patrick raised and something I\nwill tackle in the next version of this patch series.\n\nThe next version will skip all user oriented errors (errors which can be\nfixed by changing the user input), while still catch system errors (I/O\n, memory ...). I also see how this can be extended to allow users to\nnit-pick which errors they don't care about. But that is something I\ndon't plan to tackle for now.\n\n> Best Wishes\n>\n> Phillip\n\nThanks for your inputs!\n"},{"id":"515175","messageId":"20250327-245-partially-atomic-ref-updates-v5-0-4db2a3e34404@gmail.com","threadId":"62915","inReplyTo":"20250207-245-partially-atomic-ref-updates-v1-0-e6a3690ff23a@gmail.com","subject":"[PATCH v5 0/8] refs: introduce support for batched reference updates","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-03-27T11:13:24Z","receivedAt":"2025-03-27T11:13:32Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Git supports making reference updates with or without transactions.\nUpdates with transactions are generally better optimized. But\ntransactions are all or nothing. This means, if a user wants to batch\nupdates to take advantage of the optimizations without the hard\nrequirement that all updates must succeed, there is no way currently to\ndo so. Particularly with the reftable backend where batching multiple\nreference updates is more efficient than performing them sequentially.\n\nThis series introduces support for batched reference updates without\ntransactions allowing individual reference updates to fail while letting\nothers proceed. This capability is exposed through git-update-ref's\n`--allow-partial` flag, which can be used in `--stdin` mode to batch\nupdates and handle failures gracefully. Under the hood, these batched\nupdates still use the transactions infrastructure, while modifying\nsections to allow partial failures.\n\nThe changes are structured to carefully build up this functionality:\n\nFirst, we clean up and consolidate the reference update checking logic.\nThis includes removing duplicate checks in the files backend and moving\nrefname tracking to the generic layer, which simplifies the codebase and\nprepares it for the new feature.\n\nWe then restructure the reftable backend's transaction preparation code,\nextracting the update validation logic into a dedicated function. This\nnot only improves code organization but sets the stage for implementing\npartial transaction support.\n\nTo ensure we only skip errors which are user-oriented, we introduce\ntyped errors for transactions with 'enum ref_transaction_error'. We\nextend the existing errors to include other scenarios and use this new\nerrors throughout the refs code.\n\nWith this groundwork in place, we implement the core batch update\nsupport in the refs subsystem. This adds the necessary infrastructure to\ntrack and report rejected updates while allowing transactions to\nproceed. All reference backends are modified to support this behavior\nwhen enabled.\n\nFinally, we expose this functionality to users through\ngit-update-ref(1)'s `--allow-partial` flag, complete with test coverage\nand documentation. The flag is specifically limited to `--stdin` mode\nwhere batching multiple updates is most relevant.\n\nThis enhancement improves Git's flexibility in handling reference\nupdates while maintaining the safety of atomic transactions by default.\nIt's particularly valuable for tools and workflows that need to handle\nreference update failures gracefully without abandoning the entire batch\nof updates.\n\nThis series is based on top of 683c54c999 (Git 2.49, 2025-03-14) with\nPatrick's series 'refs: batch refname availability checks' [1] merged\nin.\n\n[1]: https://lore.kernel.org/all/20250217-pks-update-ref-optimization-v1-0-a2b6d87a24af@pks.im/\n\n---\nChanges in v5:\n- Inline the comments around the 'ref_transaction_error'.\n- Use 'strbuf_reset()' wherever possible instead of 'strbuf_setlen(err, 0)'.\n- Use an extra 'conflicting_dirnames' strset in 'refs_verify_refnames_available()' to track\n dirnames which were found to be conflicting, this is to avoid re-reading those dirnames.\n- Add curly braces style mismatch in if..else block.\n- Link to v4: https://lore.kernel.org/r/20250320-245-partially-atomic-ref-updates-v4-0-3dcc1b311dc9@gmail.com\n\nChanges in v4:\n- Rebased on top of 2.49 since there was a long time between the\n previous iteration and we have a new release.\n- Changed the naming to say 'batched' updates instead of 'partial\n transactions'. While we still use the transaction infrastructure\n underneath, the new naming causes less ambiguity.\n- Clean up some of the commit messages.\n- Raise BUG for invalid update index while setting rejections.\n- Fix an incorrect early return.\n- Link to v3: https://lore.kernel.org/r/20250305-245-partially-atomic-ref-updates-v3-0-0c64e3052354@gmail.com\n\nChanges in v3:\n- Changed 'transaction_error' to 'ref_transaction_error' along with the\n error names. Removed 'TRANSACTION_OK' since it can potentially be\n missed instead of simply 'return 0'.\n- Rename 'ref_transaction_set_rejected' to\n 'ref_transaction_maybe_set_rejected' and move logic around error\n checks to within this function.\n- Add a new struct 'ref_transaction_rejections' to track the rejections\n within a transaction. This allows us to only iterate over rejected\n updates.\n- Add a new commit to also support partial transactions within the\n batched F/D checks.\n- Remove NUL delimited outputs in 'git-update-ref(1)'.\n- Remove translations for plumbing outputs.\n- Other small cleanups in the commit message and code.\n\nChanges in v2:\n- Introduce and use structured errors. This consolidates the errors\n and their handling between the ref backends.\n- In the previous version, we skipped over all failures. This include\n system failures such as low memory or IO problems. Let's instead, only\n skip user-oriented failures, such as invalid old OID and so on.\n- Change the rejection function name to `ref_transaction_set_rejected()`.\n- Modify the commit messages and documentation to be a little more\n verbose.\n- Link to v1: https://lore.kernel.org/r/20250207-245-partially-atomic-ref-updates-v1-0-e6a3690ff23a@gmail.com\n\n---\n Documentation/git-update-ref.adoc | 14 +-\n builtin/fetch.c | 2 +-\n builtin/update-ref.c | 66 ++++-\n refs.c | 171 +++++++++++--\n refs.h | 70 ++++--\n refs/files-backend.c | 314 +++++++++++-------------\n refs/packed-backend.c | 69 +++---\n refs/refs-internal.h | 51 +++-\n refs/reftable-backend.c | 502 +++++++++++++++++++-------------------\n t/t1400-update-ref.sh | 233 ++++++++++++++++++\n 10 files changed, 969 insertions(+), 523 deletions(-)\n\nKarthik Nayak (8):\n refs/files: remove redundant check in split_symref_update()\n refs: move duplicate refname update check to generic layer\n refs/files: remove duplicate duplicates check\n refs/reftable: extract code from the transaction preparation\n refs: introduce enum-based transaction error types\n refs: implement batch reference update support\n refs: support rejection in batch updates during F/D checks\n update-ref: add --batch-updates flag for stdin mode\n---\n\nRange-diff versus v4:\n\n1: c682fce9d0 = 1: cae24142a1 refs/files: remove redundant check in split_symref_update()\n2: 7483120888 = 2: 239aecdb0f refs: move duplicate refname update check to generic layer\n3: e54c9042b5 = 3: 06404dd350 refs/files: remove duplicate duplicates check\n4: 4f905880af = 4: a3e645aa37 refs/reftable: extract code from the transaction preparation\n5: 7846fc43f5 ! 5: 2615bfe78e refs: introduce enum-based transaction error types\n @@ refs.h: struct worktree;\n enum ref_storage_format ref_storage_format_by_name(const char *name);\n const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format);\n \n -+/*\n -+ * enum ref_transaction_error represents the following return codes:\n -+ * REF_TRANSACTION_ERROR_GENERIC error_code: default error code.\n -+ * REF_TRANSACTION_ERROR_NAME_CONFLICT error_code: ref name conflict like A vs A/B.\n -+ * REF_TRANSACTION_ERROR_CREATE_EXISTS error_code: ref to be created already exists.\n -+ * REF_TRANSACTION_ERROR_NONEXISTENT_REF error_code: ref expected but doesn't exist.\n -+ * REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE error_code: provided old_oid or old_target of\n -+ * reference doesn't match actual.\n -+ * REF_TRANSACTION_ERROR_INVALID_NEW_VALUE error_code: provided new_oid or new_target is\n -+ * invalid.\n -+ * REF_TRANSACTION_ERROR_EXPECTED_SYMREF error_code: expected ref to be symref, but is a\n -+ * regular ref.\n -+ */\n +enum ref_transaction_error {\n ++\t/* Default error code */\n +\tREF_TRANSACTION_ERROR_GENERIC = -1,\n ++\t/* Ref name conflict like A vs A/B */\n +\tREF_TRANSACTION_ERROR_NAME_CONFLICT = -2,\n ++\t/* Ref to be created already exists */\n +\tREF_TRANSACTION_ERROR_CREATE_EXISTS = -3,\n ++\t/* ref expected but doesn't exist */\n +\tREF_TRANSACTION_ERROR_NONEXISTENT_REF = -4,\n ++\t/* Provided old_oid or old_target of reference doesn't match actual */\n +\tREF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE = -5,\n ++\t/* Provided new_oid or new_target is invalid */\n +\tREF_TRANSACTION_ERROR_INVALID_NEW_VALUE = -6,\n ++\t/* Expected ref to be symref, but is a regular ref */\n +\tREF_TRANSACTION_ERROR_EXPECTED_SYMREF = -7,\n +};\n +\n6: 398b93689a ! 6: d5c1c77b0d refs: implement batch reference update support\n @@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref\n -\t\tif (ret)\n +\t\tif (ret) {\n +\t\t\tif (ref_transaction_maybe_set_rejected(transaction, i, ret)) {\n -+\t\t\t\tstrbuf_setlen(err, 0);\n ++\t\t\t\tstrbuf_reset(err);\n +\t\t\t\tret = 0;\n +\n +\t\t\t\tcontinue;\n @@ refs/packed-backend.c: static enum ref_transaction_error write_with_updates(stru\n \t\t\t\t\tret = REF_TRANSACTION_ERROR_CREATE_EXISTS;\n +\n +\t\t\t\t\tif (ref_transaction_maybe_set_rejected(transaction, i, ret)) {\n -+\t\t\t\t\t\tstrbuf_setlen(err, 0);\n ++\t\t\t\t\t\tstrbuf_reset(err);\n +\t\t\t\t\t\tret = 0;\n +\t\t\t\t\t\tcontinue;\n +\t\t\t\t\t}\n @@ refs/packed-backend.c: static enum ref_transaction_error write_with_updates(stru\n \t\t\t\t\tret = REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;\n +\n +\t\t\t\t\tif (ref_transaction_maybe_set_rejected(transaction, i, ret)) {\n -+\t\t\t\t\t\tstrbuf_setlen(err, 0);\n ++\t\t\t\t\t\tstrbuf_reset(err);\n +\t\t\t\t\t\tret = 0;\n +\t\t\t\t\t\tcontinue;\n +\t\t\t\t\t}\n @@ refs/packed-backend.c: static enum ref_transaction_error write_with_updates(stru\n \t\t\t\tret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n +\n +\t\t\t\tif (ref_transaction_maybe_set_rejected(transaction, i, ret)) {\n -+\t\t\t\t\tstrbuf_setlen(err, 0);\n ++\t\t\t\t\tstrbuf_reset(err);\n +\t\t\t\t\tret = 0;\n +\t\t\t\t\tcontinue;\n +\t\t\t\t}\n @@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s\n -\t\tif (ret)\n +\t\tif (ret) {\n +\t\t\tif (ref_transaction_maybe_set_rejected(transaction, i, ret)) {\n -+\t\t\t\tstrbuf_setlen(err, 0);\n ++\t\t\t\tstrbuf_reset(err);\n +\t\t\t\tret = 0;\n +\n +\t\t\t\tcontinue;\n7: 965cd76097 ! 7: 4bb4902631 refs: support rejection in batch updates during F/D checks\n @@ refs.c: enum ref_transaction_error refs_verify_refnames_available(struct ref_sto\n \t\t\t\t\t struct strbuf *err)\n {\n @@ refs.c: enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs\n + \tstruct strbuf referent = STRBUF_INIT;\n + \tstruct string_list_item *item;\n + \tstruct ref_iterator *iter = NULL;\n ++\tstruct strset conflicting_dirnames;\n + \tstruct strset dirnames;\n + \tint ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;\n + \n +@@ refs.c: enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs\n + \n + \tassert(err);\n + \n ++\tstrset_init(&conflicting_dirnames);\n \tstrset_init(&dirnames);\n \n \tfor_each_string_list_item(item, refnames) {\n @@ refs.c: enum ref_transaction_error refs_verify_refnames_available(struct ref_sto\n \t\tconst char *extra_refname;\n \t\tstruct object_id oid;\n @@ refs.c: enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs\n + \t\t\t\tcontinue;\n + \n \t\t\tif (!initial_transaction &&\n - \t\t\t !refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n - \t\t\t\t\t &type, &ignore_errno)) {\n +-\t\t\t !refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n +-\t\t\t\t\t &type, &ignore_errno)) {\n ++\t\t\t (strset_contains(&conflicting_dirnames, dirname.buf) ||\n ++\t\t\t !refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n ++\t\t\t\t\t\t &type, &ignore_errno))) {\n +\t\t\t\tif (transaction && ref_transaction_maybe_set_rejected(\n +\t\t\t\t\t transaction, *update_idx,\n +\t\t\t\t\t REF_TRANSACTION_ERROR_NAME_CONFLICT)) {\n +\t\t\t\t\tstrset_remove(&dirnames, dirname.buf);\n ++\t\t\t\t\tstrset_add(&conflicting_dirnames, dirname.buf);\n +\t\t\t\t\tcontinue;\n +\t\t\t\t}\n +\n @@ refs.c: enum ref_transaction_error refs_verify_refnames_available(struct ref_sto\n \t\t\tstrbuf_addf(err, _(\"cannot process '%s' and '%s' at the same time\"),\n \t\t\t\t refname, extra_refname);\n \t\t\tgoto cleanup;\n +@@ refs.c: enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs\n + cleanup:\n + \tstrbuf_release(&referent);\n + \tstrbuf_release(&dirname);\n ++\tstrset_clear(&conflicting_dirnames);\n + \tstrset_clear(&dirnames);\n + \tref_iterator_free(iter);\n + \treturn ret;\n @@ refs.c: enum ref_transaction_error refs_verify_refname_available(\n \t};\n \n8: ed58c67cd7 ! 8: 674630f77c update-ref: add --batch-updates flag for stdin mode\n @@ builtin/update-ref.c: int cmd_update_ref(int argc,\n -\t\tupdate_refs_stdin();\n +\t\tupdate_refs_stdin(flags);\n \t\treturn 0;\n --\t}\n -+\t} else if (flags & REF_TRANSACTION_ALLOW_FAILURE)\n ++\t} else if (flags & REF_TRANSACTION_ALLOW_FAILURE) {\n +\t\tdie(\"--batch-updates can only be used with --stdin\");\n + \t}\n \n \tif (end_null)\n - \t\tusage_with_options(git_update_ref_usage, options);\n \n ## t/t1400-update-ref.sh ##\n @@ t/t1400-update-ref.sh: do\n\n\n---\n\nbase-commit: 679c868f5fffadd1f7e8e49d4d87d745ee36ffb7\nchange-id: 20241206-245-partially-atomic-ref-updates-9fe8b080345c\n\nThanks\n- Karthik\n\n"},{"id":"515176","messageId":"20250327-245-partially-atomic-ref-updates-v5-1-4db2a3e34404@gmail.com","threadId":"62915","inReplyTo":"20250327-245-partially-atomic-ref-updates-v5-0-4db2a3e34404@gmail.com","subject":"[PATCH v5 1/8] refs/files: remove redundant check in split_symref_update()","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-03-27T11:13:25Z","receivedAt":"2025-03-27T11:13:33Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"In `split_symref_update()`, there were two checks for duplicate\nrefnames:\n\n - At the start, `string_list_has_string()` ensures the refname is not\n already in `affected_refnames`, preventing duplicates from being\n added.\n\n - After adding the refname, another check verifies whether the newly\n inserted item has a `util` value.\n\nThe second check is unnecessary because the first one guarantees that\n`string_list_insert()` will never encounter a preexisting entry.\n\nThe `item->util` field is assigned to validate that a rename doesn't\nalready exist in the list. The validation is done after the first check.\nAs this check is removed, clean up the validation and the assignment of\nthis field in `split_head_update()` and `files_transaction_prepare()`.\n\nSigned-off-by: Karthik Nayak \n---\n refs/files-backend.c | 20 +++-----------------\n 1 file changed, 3 insertions(+), 17 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex ff54a4bb7e..15559a09c5 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -2382,7 +2382,6 @@ static int split_head_update(struct ref_update *update,\n \t\t\t struct string_list *affected_refnames,\n \t\t\t struct strbuf *err)\n {\n-\tstruct string_list_item *item;\n \tstruct ref_update *new_update;\n \n \tif ((update->flags & REF_LOG_ONLY) ||\n@@ -2421,8 +2420,7 @@ static int split_head_update(struct ref_update *update,\n \t */\n \tif (strcmp(new_update->refname, \"HEAD\"))\n \t\tBUG(\"%s unexpectedly not 'HEAD'\", new_update->refname);\n-\titem = string_list_insert(affected_refnames, new_update->refname);\n-\titem->util = new_update;\n+\tstring_list_insert(affected_refnames, new_update->refname);\n \n \treturn 0;\n }\n@@ -2441,7 +2439,6 @@ static int split_symref_update(struct ref_update *update,\n \t\t\t struct string_list *affected_refnames,\n \t\t\t struct strbuf *err)\n {\n-\tstruct string_list_item *item;\n \tstruct ref_update *new_update;\n \tunsigned int new_flags;\n \n@@ -2496,11 +2493,7 @@ static int split_symref_update(struct ref_update *update,\n \t * be valid as long as affected_refnames is in use, and NOT\n \t * referent, which might soon be freed by our caller.\n \t */\n-\titem = string_list_insert(affected_refnames, new_update->refname);\n-\tif (item->util)\n-\t\tBUG(\"%s unexpectedly found in affected_refnames\",\n-\t\t new_update->refname);\n-\titem->util = new_update;\n+\tstring_list_insert(affected_refnames, new_update->refname);\n \n \treturn 0;\n }\n@@ -2834,7 +2827,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t */\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n-\t\tstruct string_list_item *item;\n \n \t\tif ((update->flags & REF_IS_PRUNING) &&\n \t\t !(update->flags & REF_NO_DEREF))\n@@ -2843,13 +2835,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t\tif (update->flags & REF_LOG_ONLY)\n \t\t\tcontinue;\n \n-\t\titem = string_list_append(&affected_refnames, update->refname);\n-\t\t/*\n-\t\t * We store a pointer to update in item->util, but at\n-\t\t * the moment we never use the value of this field\n-\t\t * except to check whether it is non-NULL.\n-\t\t */\n-\t\titem->util = update;\n+\t\tstring_list_append(&affected_refnames, update->refname);\n \t}\n \tstring_list_sort(&affected_refnames);\n \tif (ref_update_reject_duplicates(&affected_refnames, err)) {\n\n-- \n2.48.1\n\n"},{"id":"515177","messageId":"20250327-245-partially-atomic-ref-updates-v5-2-4db2a3e34404@gmail.com","threadId":"62915","inReplyTo":"20250327-245-partially-atomic-ref-updates-v5-0-4db2a3e34404@gmail.com","subject":"[PATCH v5 2/8] refs: move duplicate refname update check to generic layer","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-03-27T11:13:26Z","receivedAt":"2025-03-27T11:13:34Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Move the tracking of refnames in `affected_refnames` from individual\nbackends into the generic layer in 'refs.c'. This centralizes the\nduplicate refname detection that was previously handled separately by\neach backend.\n\nMake some changes to accommodate this move:\n\n - Add a `string_list` field `refnames` to `ref_transaction` to contain\n all the references in a transaction. This field is updated whenever\n a new update is added via `ref_transaction_add_update`, so manual\n additions in reference backends are dropped.\n\n - Modify the backends to use this field internally as needed. The\n backends need to check if an update for refname already exists when\n splitting symrefs or adding an update for 'HEAD'.\n\n - In the reftable backend, within `reftable_be_transaction_prepare()`,\n move the `string_list_has_string()` check above\n `ref_transaction_add_update()`. Since `ref_transaction_add_update()`\n automatically adds the refname to `transaction->refnames`,\n performing the check after will always return true, so we perform\n the check before adding the update.\n\nThis helps reduce duplication of functionality between the backends and\nmakes it easier to make changes in a more centralized manner.\n\nSigned-off-by: Karthik Nayak \n---\n refs.c | 17 +++++++++++++\n refs/files-backend.c | 67 +++++++++++--------------------------------------\n refs/packed-backend.c | 25 +-----------------\n refs/refs-internal.h | 2 ++\n refs/reftable-backend.c | 54 +++++++++++++--------------------------\n 5 files changed, 51 insertions(+), 114 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 2ac9d8ebd0..504bf2063e 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1175,6 +1175,7 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,\n \tCALLOC_ARRAY(tr, 1);\n \ttr->ref_store = refs;\n \ttr->flags = flags;\n+\tstring_list_init_dup(&tr->refnames);\n \treturn tr;\n }\n \n@@ -1205,6 +1206,7 @@ void ref_transaction_free(struct ref_transaction *transaction)\n \t\tfree((char *)transaction->updates[i]->old_target);\n \t\tfree(transaction->updates[i]);\n \t}\n+\tstring_list_clear(&transaction->refnames, 0);\n \tfree(transaction->updates);\n \tfree(transaction);\n }\n@@ -1218,6 +1220,7 @@ struct ref_update *ref_transaction_add_update(\n \t\tconst char *committer_info,\n \t\tconst char *msg)\n {\n+\tstruct string_list_item *item;\n \tstruct ref_update *update;\n \n \tif (transaction->state != REF_TRANSACTION_OPEN)\n@@ -1245,6 +1248,16 @@ struct ref_update *ref_transaction_add_update(\n \t\tupdate->msg = normalize_reflog_message(msg);\n \t}\n \n+\t/*\n+\t * This list is generally used by the backends to avoid duplicates.\n+\t * But we do support multiple log updates for a given refname within\n+\t * a single transaction.\n+\t */\n+\tif (!(update->flags & REF_LOG_ONLY)) {\n+\t\titem = string_list_append(&transaction->refnames, refname);\n+\t\titem->util = update;\n+\t}\n+\n \treturn update;\n }\n \n@@ -2405,6 +2418,10 @@ int ref_transaction_prepare(struct ref_transaction *transaction,\n \t\treturn -1;\n \t}\n \n+\tstring_list_sort(&transaction->refnames);\n+\tif (ref_update_reject_duplicates(&transaction->refnames, err))\n+\t\treturn TRANSACTION_GENERIC_ERROR;\n+\n \tret = refs->be->transaction_prepare(refs, transaction, err);\n \tif (ret)\n \t\treturn ret;\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 15559a09c5..58f62ea8a3 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -2378,9 +2378,7 @@ static struct ref_iterator *files_reflog_iterator_begin(struct ref_store *ref_st\n */\n static int split_head_update(struct ref_update *update,\n \t\t\t struct ref_transaction *transaction,\n-\t\t\t const char *head_ref,\n-\t\t\t struct string_list *affected_refnames,\n-\t\t\t struct strbuf *err)\n+\t\t\t const char *head_ref, struct strbuf *err)\n {\n \tstruct ref_update *new_update;\n \n@@ -2398,7 +2396,7 @@ static int split_head_update(struct ref_update *update,\n \t * transaction. This check is O(lg N) in the transaction\n \t * size, but it happens at most once per transaction.\n \t */\n-\tif (string_list_has_string(affected_refnames, \"HEAD\")) {\n+\tif (string_list_has_string(&transaction->refnames, \"HEAD\")) {\n \t\t/* An entry already existed */\n \t\tstrbuf_addf(err,\n \t\t\t \"multiple updates for 'HEAD' (including one \"\n@@ -2420,7 +2418,6 @@ static int split_head_update(struct ref_update *update,\n \t */\n \tif (strcmp(new_update->refname, \"HEAD\"))\n \t\tBUG(\"%s unexpectedly not 'HEAD'\", new_update->refname);\n-\tstring_list_insert(affected_refnames, new_update->refname);\n \n \treturn 0;\n }\n@@ -2436,7 +2433,6 @@ static int split_head_update(struct ref_update *update,\n static int split_symref_update(struct ref_update *update,\n \t\t\t const char *referent,\n \t\t\t struct ref_transaction *transaction,\n-\t\t\t struct string_list *affected_refnames,\n \t\t\t struct strbuf *err)\n {\n \tstruct ref_update *new_update;\n@@ -2448,7 +2444,7 @@ static int split_symref_update(struct ref_update *update,\n \t * size, but it happens at most once per symref in a\n \t * transaction.\n \t */\n-\tif (string_list_has_string(affected_refnames, referent)) {\n+\tif (string_list_has_string(&transaction->refnames, referent)) {\n \t\t/* An entry already exists */\n \t\tstrbuf_addf(err,\n \t\t\t \"multiple updates for '%s' (including one \"\n@@ -2486,15 +2482,6 @@ static int split_symref_update(struct ref_update *update,\n \tupdate->flags |= REF_LOG_ONLY | REF_NO_DEREF;\n \tupdate->flags &= ~REF_HAVE_OLD;\n \n-\t/*\n-\t * Add the referent. This insertion is O(N) in the transaction\n-\t * size, but it happens at most once per symref in a\n-\t * transaction. Make sure to add new_update->refname, which will\n-\t * be valid as long as affected_refnames is in use, and NOT\n-\t * referent, which might soon be freed by our caller.\n-\t */\n-\tstring_list_insert(affected_refnames, new_update->refname);\n-\n \treturn 0;\n }\n \n@@ -2558,7 +2545,6 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\t\t struct ref_transaction *transaction,\n \t\t\t const char *head_ref,\n \t\t\t struct string_list *refnames_to_check,\n-\t\t\t struct string_list *affected_refnames,\n \t\t\t struct strbuf *err)\n {\n \tstruct strbuf referent = STRBUF_INIT;\n@@ -2575,8 +2561,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\tupdate->flags |= REF_DELETING;\n \n \tif (head_ref) {\n-\t\tret = split_head_update(update, transaction, head_ref,\n-\t\t\t\t\taffected_refnames, err);\n+\t\tret = split_head_update(update, transaction, head_ref, err);\n \t\tif (ret)\n \t\t\tgoto out;\n \t}\n@@ -2586,9 +2571,8 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\tlock->count++;\n \t} else {\n \t\tret = lock_raw_ref(refs, update->refname, mustexist,\n-\t\t\t\t refnames_to_check, affected_refnames,\n-\t\t\t\t &lock, &referent,\n-\t\t\t\t &update->type, err);\n+\t\t\t\t refnames_to_check, &transaction->refnames,\n+\t\t\t\t &lock, &referent, &update->type, err);\n \t\tif (ret) {\n \t\t\tchar *reason;\n \n@@ -2642,9 +2626,8 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\t\t * of processing the split-off update, so we\n \t\t\t * don't have to do it here.\n \t\t\t */\n-\t\t\tret = split_symref_update(update,\n-\t\t\t\t\t\t referent.buf, transaction,\n-\t\t\t\t\t\t affected_refnames, err);\n+\t\t\tret = split_symref_update(update, referent.buf,\n+\t\t\t\t\t\t transaction, err);\n \t\t\tif (ret)\n \t\t\t\tgoto out;\n \t\t}\n@@ -2799,7 +2782,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t\t\t \"ref_transaction_prepare\");\n \tsize_t i;\n \tint ret = 0;\n-\tstruct string_list affected_refnames = STRING_LIST_INIT_NODUP;\n \tstruct string_list refnames_to_check = STRING_LIST_INIT_NODUP;\n \tchar *head_ref = NULL;\n \tint head_type;\n@@ -2818,12 +2800,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \ttransaction->backend_data = backend_data;\n \n \t/*\n-\t * Fail if a refname appears more than once in the\n-\t * transaction. (If we end up splitting up any updates using\n-\t * split_symref_update() or split_head_update(), those\n-\t * functions will check that the new updates don't have the\n-\t * same refname as any existing ones.) Also fail if any of the\n-\t * updates use REF_IS_PRUNING without REF_NO_DEREF.\n+\t * Fail if any of the updates use REF_IS_PRUNING without REF_NO_DEREF.\n \t */\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n@@ -2831,16 +2808,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t\tif ((update->flags & REF_IS_PRUNING) &&\n \t\t !(update->flags & REF_NO_DEREF))\n \t\t\tBUG(\"REF_IS_PRUNING set without REF_NO_DEREF\");\n-\n-\t\tif (update->flags & REF_LOG_ONLY)\n-\t\t\tcontinue;\n-\n-\t\tstring_list_append(&affected_refnames, update->refname);\n-\t}\n-\tstring_list_sort(&affected_refnames);\n-\tif (ref_update_reject_duplicates(&affected_refnames, err)) {\n-\t\tret = TRANSACTION_GENERIC_ERROR;\n-\t\tgoto cleanup;\n \t}\n \n \t/*\n@@ -2882,7 +2849,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \n \t\tret = lock_ref_for_update(refs, update, transaction,\n \t\t\t\t\t head_ref, &refnames_to_check,\n-\t\t\t\t\t &affected_refnames, err);\n+\t\t\t\t\t err);\n \t\tif (ret)\n \t\t\tgoto cleanup;\n \n@@ -2929,7 +2896,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t * So instead, we accept the race for now.\n \t */\n \tif (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,\n-\t\t\t\t\t &affected_refnames, NULL, 0, err)) {\n+\t\t\t\t\t &transaction->refnames, NULL, 0, err)) {\n \t\tret = TRANSACTION_NAME_CONFLICT;\n \t\tgoto cleanup;\n \t}\n@@ -2975,7 +2942,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \n cleanup:\n \tfree(head_ref);\n-\tstring_list_clear(&affected_refnames, 0);\n \tstring_list_clear(&refnames_to_check, 0);\n \n \tif (ret)\n@@ -3050,13 +3016,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \tif (transaction->state != REF_TRANSACTION_PREPARED)\n \t\tBUG(\"commit called for transaction that is not prepared\");\n \n-\t/* Fail if a refname appears more than once in the transaction: */\n-\tfor (i = 0; i < transaction->nr; i++)\n-\t\tif (!(transaction->updates[i]->flags & REF_LOG_ONLY))\n-\t\t\tstring_list_append(&affected_refnames,\n-\t\t\t\t\t transaction->updates[i]->refname);\n-\tstring_list_sort(&affected_refnames);\n-\tif (ref_update_reject_duplicates(&affected_refnames, err)) {\n+\tstring_list_sort(&transaction->refnames);\n+\tif (ref_update_reject_duplicates(&transaction->refnames, err)) {\n \t\tret = TRANSACTION_GENERIC_ERROR;\n \t\tgoto cleanup;\n \t}\n@@ -3074,7 +3035,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \t * that we are creating already exists.\n \t */\n \tif (refs_for_each_rawref(&refs->base, ref_present,\n-\t\t\t\t &affected_refnames))\n+\t\t\t\t &transaction->refnames))\n \t\tBUG(\"initial ref transaction called with existing refs\");\n \n \tpacked_transaction = ref_store_transaction_begin(refs->packed_ref_store,\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex f4c82ba2c7..19220d2e99 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1622,8 +1622,6 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n struct packed_transaction_backend_data {\n \t/* True iff the transaction owns the packed-refs lock. */\n \tint own_lock;\n-\n-\tstruct string_list updates;\n };\n \n static void packed_transaction_cleanup(struct packed_ref_store *refs,\n@@ -1632,8 +1630,6 @@ static void packed_transaction_cleanup(struct packed_ref_store *refs,\n \tstruct packed_transaction_backend_data *data = transaction->backend_data;\n \n \tif (data) {\n-\t\tstring_list_clear(&data->updates, 0);\n-\n \t\tif (is_tempfile_active(refs->tempfile))\n \t\t\tdelete_tempfile(&refs->tempfile);\n \n@@ -1658,7 +1654,6 @@ static int packed_transaction_prepare(struct ref_store *ref_store,\n \t\t\tREF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,\n \t\t\t\"ref_transaction_prepare\");\n \tstruct packed_transaction_backend_data *data;\n-\tsize_t i;\n \tint ret = TRANSACTION_GENERIC_ERROR;\n \n \t/*\n@@ -1671,34 +1666,16 @@ static int packed_transaction_prepare(struct ref_store *ref_store,\n \t */\n \n \tCALLOC_ARRAY(data, 1);\n-\tstring_list_init_nodup(&data->updates);\n \n \ttransaction->backend_data = data;\n \n-\t/*\n-\t * Stick the updates in a string list by refname so that we\n-\t * can sort them:\n-\t */\n-\tfor (i = 0; i < transaction->nr; i++) {\n-\t\tstruct ref_update *update = transaction->updates[i];\n-\t\tstruct string_list_item *item =\n-\t\t\tstring_list_append(&data->updates, update->refname);\n-\n-\t\t/* Store a pointer to update in item->util: */\n-\t\titem->util = update;\n-\t}\n-\tstring_list_sort(&data->updates);\n-\n-\tif (ref_update_reject_duplicates(&data->updates, err))\n-\t\tgoto failure;\n-\n \tif (!is_lock_file_locked(&refs->lock)) {\n \t\tif (packed_refs_lock(ref_store, 0, err))\n \t\t\tgoto failure;\n \t\tdata->own_lock = 1;\n \t}\n \n-\tif (write_with_updates(refs, &data->updates, err))\n+\tif (write_with_updates(refs, &transaction->refnames, err))\n \t\tgoto failure;\n \n \ttransaction->state = REF_TRANSACTION_PREPARED;\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex e5862757a7..92db793026 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -3,6 +3,7 @@\n \n #include \"refs.h\"\n #include \"iterator.h\"\n+#include \"string-list.h\"\n \n struct fsck_options;\n struct ref_transaction;\n@@ -198,6 +199,7 @@ enum ref_transaction_state {\n struct ref_transaction {\n \tstruct ref_store *ref_store;\n \tstruct ref_update **updates;\n+\tstruct string_list refnames;\n \tsize_t alloc;\n \tsize_t nr;\n \tenum ref_transaction_state state;\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex ae434cd248..a92c9a2f4f 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -1076,7 +1076,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \tstruct reftable_ref_store *refs =\n \t\treftable_be_downcast(ref_store, REF_STORE_WRITE|REF_STORE_MAIN, \"ref_transaction_prepare\");\n \tstruct strbuf referent = STRBUF_INIT, head_referent = STRBUF_INIT;\n-\tstruct string_list affected_refnames = STRING_LIST_INIT_NODUP;\n \tstruct string_list refnames_to_check = STRING_LIST_INIT_NODUP;\n \tstruct reftable_transaction_data *tx_data = NULL;\n \tstruct reftable_backend *be;\n@@ -1101,10 +1100,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\t\t\t transaction->updates[i], err);\n \t\tif (ret)\n \t\t\tgoto done;\n-\n-\t\tif (!(transaction->updates[i]->flags & REF_LOG_ONLY))\n-\t\t\tstring_list_append(&affected_refnames,\n-\t\t\t\t\t transaction->updates[i]->refname);\n \t}\n \n \t/*\n@@ -1116,17 +1111,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\ttx_data->args[i].updates_alloc = tx_data->args[i].updates_expected;\n \t}\n \n-\t/*\n-\t * Fail if a refname appears more than once in the transaction.\n-\t * This code is taken from the files backend and is a good candidate to\n-\t * be moved into the generic layer.\n-\t */\n-\tstring_list_sort(&affected_refnames);\n-\tif (ref_update_reject_duplicates(&affected_refnames, err)) {\n-\t\tret = TRANSACTION_GENERIC_ERROR;\n-\t\tgoto done;\n-\t}\n-\n \t/*\n \t * TODO: it's dubious whether we should reload the stack that \"HEAD\"\n \t * belongs to or not. In theory, it may happen that we only modify\n@@ -1194,14 +1178,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t !(u->flags & REF_LOG_ONLY) &&\n \t\t !(u->flags & REF_UPDATE_VIA_HEAD) &&\n \t\t !strcmp(rewritten_ref, head_referent.buf)) {\n-\t\t\tstruct ref_update *new_update;\n-\n \t\t\t/*\n \t\t\t * First make sure that HEAD is not already in the\n \t\t\t * transaction. This check is O(lg N) in the transaction\n \t\t\t * size, but it happens at most once per transaction.\n \t\t\t */\n-\t\t\tif (string_list_has_string(&affected_refnames, \"HEAD\")) {\n+\t\t\tif (string_list_has_string(&transaction->refnames, \"HEAD\")) {\n \t\t\t\t/* An entry already existed */\n \t\t\t\tstrbuf_addf(err,\n \t\t\t\t\t _(\"multiple updates for 'HEAD' (including one \"\n@@ -1211,12 +1193,11 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\tgoto done;\n \t\t\t}\n \n-\t\t\tnew_update = ref_transaction_add_update(\n-\t\t\t\t\ttransaction, \"HEAD\",\n-\t\t\t\t\tu->flags | REF_LOG_ONLY | REF_NO_DEREF,\n-\t\t\t\t\t&u->new_oid, &u->old_oid, NULL, NULL, NULL,\n-\t\t\t\t\tu->msg);\n-\t\t\tstring_list_insert(&affected_refnames, new_update->refname);\n+\t\t\tref_transaction_add_update(\n+\t\t\t\ttransaction, \"HEAD\",\n+\t\t\t\tu->flags | REF_LOG_ONLY | REF_NO_DEREF,\n+\t\t\t\t&u->new_oid, &u->old_oid, NULL, NULL, NULL,\n+\t\t\t\tu->msg);\n \t\t}\n \n \t\tret = reftable_backend_read_ref(be, rewritten_ref,\n@@ -1281,6 +1262,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\tif (!strcmp(rewritten_ref, \"HEAD\"))\n \t\t\t\t\tnew_flags |= REF_UPDATE_VIA_HEAD;\n \n+\t\t\t\tif (string_list_has_string(&transaction->refnames, referent.buf)) {\n+\t\t\t\t\tstrbuf_addf(err,\n+\t\t\t\t\t\t _(\"multiple updates for '%s' (including one \"\n+\t\t\t\t\t\t \"via symref '%s') are not allowed\"),\n+\t\t\t\t\t\t referent.buf, u->refname);\n+\t\t\t\t\tret = TRANSACTION_NAME_CONFLICT;\n+\t\t\t\t\tgoto done;\n+\t\t\t\t}\n+\n \t\t\t\t/*\n \t\t\t\t * If we are updating a symref (eg. HEAD), we should also\n \t\t\t\t * update the branch that the symref points to.\n@@ -1305,16 +1295,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\t */\n \t\t\t\tu->flags |= REF_LOG_ONLY | REF_NO_DEREF;\n \t\t\t\tu->flags &= ~REF_HAVE_OLD;\n-\n-\t\t\t\tif (string_list_has_string(&affected_refnames, new_update->refname)) {\n-\t\t\t\t\tstrbuf_addf(err,\n-\t\t\t\t\t\t _(\"multiple updates for '%s' (including one \"\n-\t\t\t\t\t\t \"via symref '%s') are not allowed\"),\n-\t\t\t\t\t\t referent.buf, u->refname);\n-\t\t\t\t\tret = TRANSACTION_NAME_CONFLICT;\n-\t\t\t\t\tgoto done;\n-\t\t\t\t}\n-\t\t\t\tstring_list_insert(&affected_refnames, new_update->refname);\n \t\t\t}\n \t\t}\n \n@@ -1383,7 +1363,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t}\n \t}\n \n-\tret = refs_verify_refnames_available(ref_store, &refnames_to_check, &affected_refnames, NULL,\n+\tret = refs_verify_refnames_available(ref_store, &refnames_to_check,\n+\t\t\t\t\t &transaction->refnames, NULL,\n \t\t\t\t\t transaction->flags & REF_TRANSACTION_FLAG_INITIAL,\n \t\t\t\t\t err);\n \tif (ret < 0)\n@@ -1401,7 +1382,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\tstrbuf_addf(err, _(\"reftable: transaction prepare: %s\"),\n \t\t\t\t reftable_error_str(ret));\n \t}\n-\tstring_list_clear(&affected_refnames, 0);\n \tstrbuf_release(&referent);\n \tstrbuf_release(&head_referent);\n \tstring_list_clear(&refnames_to_check, 0);\n\n-- \n2.48.1\n\n"},{"id":"515178","messageId":"20250327-245-partially-atomic-ref-updates-v5-3-4db2a3e34404@gmail.com","threadId":"62915","inReplyTo":"20250327-245-partially-atomic-ref-updates-v5-0-4db2a3e34404@gmail.com","subject":"[PATCH v5 3/8] refs/files: remove duplicate duplicates check","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-03-27T11:13:27Z","receivedAt":"2025-03-27T11:13:35Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Within the files reference backend's transaction's 'finish' phase, a\nverification step is currently performed wherein the refnames list is\nsorted and examined for multiple updates targeting the same refname.\n\nIt has been observed that this verification is redundant, as an\nidentical check is already executed during the transaction's 'prepare'\nstage. Since the refnames list remains unmodified following the\n'prepare' stage, this secondary verification can be safely eliminated.\n\nThe duplicate check has been removed accordingly, and the\n`ref_update_reject_duplicates()` function has been marked as static, as\nits usage is now confined to 'refs.c'.\n\nSigned-off-by: Karthik Nayak \n---\n refs.c | 9 +++++++--\n refs/files-backend.c | 6 ------\n refs/refs-internal.h | 8 --------\n 3 files changed, 7 insertions(+), 16 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 504bf2063e..61bed9672a 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2303,8 +2303,13 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,\n \treturn ret;\n }\n \n-int ref_update_reject_duplicates(struct string_list *refnames,\n-\t\t\t\t struct strbuf *err)\n+/*\n+ * Write an error to `err` and return a nonzero value iff the same\n+ * refname appears multiple times in `refnames`. `refnames` must be\n+ * sorted on entry to this function.\n+ */\n+static int ref_update_reject_duplicates(struct string_list *refnames,\n+\t\t\t\t\tstruct strbuf *err)\n {\n \tsize_t i, n = refnames->nr;\n \ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 58f62ea8a3..ea023a59fc 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3016,12 +3016,6 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \tif (transaction->state != REF_TRANSACTION_PREPARED)\n \t\tBUG(\"commit called for transaction that is not prepared\");\n \n-\tstring_list_sort(&transaction->refnames);\n-\tif (ref_update_reject_duplicates(&transaction->refnames, err)) {\n-\t\tret = TRANSACTION_GENERIC_ERROR;\n-\t\tgoto cleanup;\n-\t}\n-\n \t/*\n \t * It's really undefined to call this function in an active\n \t * repository or when there are existing references: we are\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 92db793026..6d3770d0cc 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -142,14 +142,6 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t struct object_id *oid, struct strbuf *referent,\n \t\t unsigned int *type, int *failure_errno);\n \n-/*\n- * Write an error to `err` and return a nonzero value iff the same\n- * refname appears multiple times in `refnames`. `refnames` must be\n- * sorted on entry to this function.\n- */\n-int ref_update_reject_duplicates(struct string_list *refnames,\n-\t\t\t\t struct strbuf *err);\n-\n /*\n * Add a ref_update with the specified properties to transaction, and\n * return a pointer to the new object. This function does not verify\n\n-- \n2.48.1\n\n"},{"id":"515179","messageId":"20250327-245-partially-atomic-ref-updates-v5-4-4db2a3e34404@gmail.com","threadId":"62915","inReplyTo":"20250327-245-partially-atomic-ref-updates-v5-0-4db2a3e34404@gmail.com","subject":"[PATCH v5 4/8] refs/reftable: extract code from the transaction preparation","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-03-27T11:13:28Z","receivedAt":"2025-03-27T11:13:36Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Extract the core logic for preparing individual reference updates from\n`reftable_be_transaction_prepare()` into `prepare_single_update()`. This\ndedicated function now handles all validation and preparation steps for\neach reference update in the transaction, including object ID\nverification, HEAD reference handling, and symref processing.\n\nThe refactoring consolidates all reference update validation into a\nsingle logical block, which improves code maintainability and\nreadability. More importantly, this restructuring lays the groundwork\nfor implementing batched reference update support in the reftable\nbackend, which will be introduced in a followup commit.\n\nNo functional changes are included in this commit - it is purely a code\nreorganization to support future enhancements.\n\nSigned-off-by: Karthik Nayak \n---\n refs/reftable-backend.c | 463 +++++++++++++++++++++++++-----------------------\n 1 file changed, 237 insertions(+), 226 deletions(-)\n\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex a92c9a2f4f..786df11a03 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -1069,6 +1069,239 @@ static int queue_transaction_update(struct reftable_ref_store *refs,\n \treturn 0;\n }\n \n+static int prepare_single_update(struct reftable_ref_store *refs,\n+\t\t\t\t struct reftable_transaction_data *tx_data,\n+\t\t\t\t struct ref_transaction *transaction,\n+\t\t\t\t struct reftable_backend *be,\n+\t\t\t\t struct ref_update *u,\n+\t\t\t\t struct string_list *refnames_to_check,\n+\t\t\t\t unsigned int head_type,\n+\t\t\t\t struct strbuf *head_referent,\n+\t\t\t\t struct strbuf *referent,\n+\t\t\t\t struct strbuf *err)\n+{\n+\tstruct object_id current_oid = {0};\n+\tconst char *rewritten_ref;\n+\tint ret = 0;\n+\n+\t/*\n+\t * There is no need to reload the respective backends here as\n+\t * we have already reloaded them when preparing the transaction\n+\t * update. And given that the stacks have been locked there\n+\t * shouldn't have been any concurrent modifications of the\n+\t * stack.\n+\t */\n+\tret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\t/* Verify that the new object ID is valid. */\n+\tif ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&\n+\t !(u->flags & REF_SKIP_OID_VERIFICATION) &&\n+\t !(u->flags & REF_LOG_ONLY)) {\n+\t\tstruct object *o = parse_object(refs->base.repo, &u->new_oid);\n+\t\tif (!o) {\n+\t\t\tstrbuf_addf(err,\n+\t\t\t\t _(\"trying to write ref '%s' with nonexistent object %s\"),\n+\t\t\t\t u->refname, oid_to_hex(&u->new_oid));\n+\t\t\treturn -1;\n+\t\t}\n+\n+\t\tif (o->type != OBJ_COMMIT && is_branch(u->refname)) {\n+\t\t\tstrbuf_addf(err, _(\"trying to write non-commit object %s to branch '%s'\"),\n+\t\t\t\t oid_to_hex(&u->new_oid), u->refname);\n+\t\t\treturn -1;\n+\t\t}\n+\t}\n+\n+\t/*\n+\t * When we update the reference that HEAD points to we enqueue\n+\t * a second log-only update for HEAD so that its reflog is\n+\t * updated accordingly.\n+\t */\n+\tif (head_type == REF_ISSYMREF &&\n+\t !(u->flags & REF_LOG_ONLY) &&\n+\t !(u->flags & REF_UPDATE_VIA_HEAD) &&\n+\t !strcmp(rewritten_ref, head_referent->buf)) {\n+\t\t/*\n+\t\t * First make sure that HEAD is not already in the\n+\t\t * transaction. This check is O(lg N) in the transaction\n+\t\t * size, but it happens at most once per transaction.\n+\t\t */\n+\t\tif (string_list_has_string(&transaction->refnames, \"HEAD\")) {\n+\t\t\t/* An entry already existed */\n+\t\t\tstrbuf_addf(err,\n+\t\t\t\t _(\"multiple updates for 'HEAD' (including one \"\n+\t\t\t\t \"via its referent '%s') are not allowed\"),\n+\t\t\t\t u->refname);\n+\t\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t\t}\n+\n+\t\tref_transaction_add_update(\n+\t\t\ttransaction, \"HEAD\",\n+\t\t\tu->flags | REF_LOG_ONLY | REF_NO_DEREF,\n+\t\t\t&u->new_oid, &u->old_oid, NULL, NULL, NULL,\n+\t\t\tu->msg);\n+\t}\n+\n+\tret = reftable_backend_read_ref(be, rewritten_ref,\n+\t\t\t\t\t¤t_oid, referent, &u->type);\n+\tif (ret < 0)\n+\t\treturn ret;\n+\tif (ret > 0 && !ref_update_expects_existing_old_ref(u)) {\n+\t\t/*\n+\t\t * The reference does not exist, and we either have no\n+\t\t * old object ID or expect the reference to not exist.\n+\t\t * We can thus skip below safety checks as well as the\n+\t\t * symref splitting. But we do want to verify that\n+\t\t * there is no conflicting reference here so that we\n+\t\t * can output a proper error message instead of failing\n+\t\t * at a later point.\n+\t\t */\n+\t\tstring_list_append(refnames_to_check, u->refname);\n+\n+\t\t/*\n+\t\t * There is no need to write the reference deletion\n+\t\t * when the reference in question doesn't exist.\n+\t\t */\n+\t\tif ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {\n+\t\t\tret = queue_transaction_update(refs, tx_data, u,\n+\t\t\t\t\t\t ¤t_oid, err);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\t\t}\n+\n+\t\treturn 0;\n+\t}\n+\tif (ret > 0) {\n+\t\t/* The reference does not exist, but we expected it to. */\n+\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n+\n+\n+\t\t\t\t \"unable to resolve reference '%s'\"),\n+\t\t\t ref_update_original_update_refname(u), u->refname);\n+\t\treturn -1;\n+\t}\n+\n+\tif (u->type & REF_ISSYMREF) {\n+\t\t/*\n+\t\t * The reftable stack is locked at this point already,\n+\t\t * so it is safe to call `refs_resolve_ref_unsafe()`\n+\t\t * here without causing races.\n+\t\t */\n+\t\tconst char *resolved = refs_resolve_ref_unsafe(&refs->base, u->refname, 0,\n+\t\t\t\t\t\t\t ¤t_oid, NULL);\n+\n+\t\tif (u->flags & REF_NO_DEREF) {\n+\t\t\tif (u->flags & REF_HAVE_OLD && !resolved) {\n+\t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n+\t\t\t\t\t\t \"error reading reference\"), u->refname);\n+\t\t\t\treturn -1;\n+\t\t\t}\n+\t\t} else {\n+\t\t\tstruct ref_update *new_update;\n+\t\t\tint new_flags;\n+\n+\t\t\tnew_flags = u->flags;\n+\t\t\tif (!strcmp(rewritten_ref, \"HEAD\"))\n+\t\t\t\tnew_flags |= REF_UPDATE_VIA_HEAD;\n+\n+\t\t\tif (string_list_has_string(&transaction->refnames, referent->buf)) {\n+\t\t\t\tstrbuf_addf(err,\n+\t\t\t\t\t _(\"multiple updates for '%s' (including one \"\n+\t\t\t\t\t \"via symref '%s') are not allowed\"),\n+\t\t\t\t\t referent->buf, u->refname);\n+\t\t\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t\t\t}\n+\n+\t\t\t/*\n+\t\t\t * If we are updating a symref (eg. HEAD), we should also\n+\t\t\t * update the branch that the symref points to.\n+\t\t\t *\n+\t\t\t * This is generic functionality, and would be better\n+\t\t\t * done in refs.c, but the current implementation is\n+\t\t\t * intertwined with the locking in files-backend.c.\n+\t\t\t */\n+\t\t\tnew_update = ref_transaction_add_update(\n+\t\t\t\ttransaction, referent->buf, new_flags,\n+\t\t\t\tu->new_target ? NULL : &u->new_oid,\n+\t\t\t\tu->old_target ? NULL : &u->old_oid,\n+\t\t\t\tu->new_target, u->old_target,\n+\t\t\t\tu->committer_info, u->msg);\n+\n+\t\t\tnew_update->parent_update = u;\n+\n+\t\t\t/*\n+\t\t\t * Change the symbolic ref update to log only. Also, it\n+\t\t\t * doesn't need to check its old OID value, as that will be\n+\t\t\t * done when new_update is processed.\n+\t\t\t */\n+\t\t\tu->flags |= REF_LOG_ONLY | REF_NO_DEREF;\n+\t\t\tu->flags &= ~REF_HAVE_OLD;\n+\t\t}\n+\t}\n+\n+\t/*\n+\t * Verify that the old object matches our expectations. Note\n+\t * that the error messages here do not make a lot of sense in\n+\t * the context of the reftable backend as we never lock\n+\t * individual refs. But the error messages match what the files\n+\t * backend returns, which keeps our tests happy.\n+\t */\n+\tif (u->old_target) {\n+\t\tif (!(u->type & REF_ISSYMREF)) {\n+\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n+\t\t\t\t\t \"expected symref with target '%s': \"\n+\t\t\t\t\t \"but is a regular ref\"),\n+\t\t\t\t ref_update_original_update_refname(u),\n+\t\t\t\t u->old_target);\n+\t\t\treturn -1;\n+\t\t}\n+\n+\t\tif (ref_update_check_old_target(referent->buf, u, err)) {\n+\t\t\treturn -1;\n+\t\t}\n+\t} else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {\n+\t\tif (is_null_oid(&u->old_oid)) {\n+\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n+\t\t\t\t\t \"reference already exists\"),\n+\t\t\t\t ref_update_original_update_refname(u));\n+\t\t\treturn TRANSACTION_CREATE_EXISTS;\n+\t\t}\n+\t\telse if (is_null_oid(¤t_oid))\n+\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n+\t\t\t\t\t \"reference is missing but expected %s\"),\n+\t\t\t\t ref_update_original_update_refname(u),\n+\t\t\t\t oid_to_hex(&u->old_oid));\n+\t\telse\n+\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n+\t\t\t\t\t \"is at %s but expected %s\"),\n+\t\t\t\t ref_update_original_update_refname(u),\n+\t\t\t\t oid_to_hex(¤t_oid),\n+\t\t\t\t oid_to_hex(&u->old_oid));\n+\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t}\n+\n+\t/*\n+\t * If all of the following conditions are true:\n+\t *\n+\t * - We're not about to write a symref.\n+\t * - We're not about to write a log-only entry.\n+\t * - Old and new object ID are different.\n+\t *\n+\t * Then we're essentially doing a no-op update that can be\n+\t * skipped. This is not only for the sake of efficiency, but\n+\t * also skips writing unneeded reflog entries.\n+\t */\n+\tif ((u->type & REF_ISSYMREF) ||\n+\t (u->flags & REF_LOG_ONLY) ||\n+\t (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid)))\n+\t\treturn queue_transaction_update(refs, tx_data, u,\n+\t\t\t\t\t ¤t_oid, err);\n+\n+\treturn 0;\n+}\n+\n static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\t\t struct ref_transaction *transaction,\n \t\t\t\t\t struct strbuf *err)\n@@ -1133,234 +1366,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \tret = 0;\n \n \tfor (i = 0; i < transaction->nr; i++) {\n-\t\tstruct ref_update *u = transaction->updates[i];\n-\t\tstruct object_id current_oid = {0};\n-\t\tconst char *rewritten_ref;\n-\n-\t\t/*\n-\t\t * There is no need to reload the respective backends here as\n-\t\t * we have already reloaded them when preparing the transaction\n-\t\t * update. And given that the stacks have been locked there\n-\t\t * shouldn't have been any concurrent modifications of the\n-\t\t * stack.\n-\t\t */\n-\t\tret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);\n+\t\tret = prepare_single_update(refs, tx_data, transaction, be,\n+\t\t\t\t\t transaction->updates[i],\n+\t\t\t\t\t &refnames_to_check, head_type,\n+\t\t\t\t\t &head_referent, &referent, err);\n \t\tif (ret)\n \t\t\tgoto done;\n-\n-\t\t/* Verify that the new object ID is valid. */\n-\t\tif ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&\n-\t\t !(u->flags & REF_SKIP_OID_VERIFICATION) &&\n-\t\t !(u->flags & REF_LOG_ONLY)) {\n-\t\t\tstruct object *o = parse_object(refs->base.repo, &u->new_oid);\n-\t\t\tif (!o) {\n-\t\t\t\tstrbuf_addf(err,\n-\t\t\t\t\t _(\"trying to write ref '%s' with nonexistent object %s\"),\n-\t\t\t\t\t u->refname, oid_to_hex(&u->new_oid));\n-\t\t\t\tret = -1;\n-\t\t\t\tgoto done;\n-\t\t\t}\n-\n-\t\t\tif (o->type != OBJ_COMMIT && is_branch(u->refname)) {\n-\t\t\t\tstrbuf_addf(err, _(\"trying to write non-commit object %s to branch '%s'\"),\n-\t\t\t\t\t oid_to_hex(&u->new_oid), u->refname);\n-\t\t\t\tret = -1;\n-\t\t\t\tgoto done;\n-\t\t\t}\n-\t\t}\n-\n-\t\t/*\n-\t\t * When we update the reference that HEAD points to we enqueue\n-\t\t * a second log-only update for HEAD so that its reflog is\n-\t\t * updated accordingly.\n-\t\t */\n-\t\tif (head_type == REF_ISSYMREF &&\n-\t\t !(u->flags & REF_LOG_ONLY) &&\n-\t\t !(u->flags & REF_UPDATE_VIA_HEAD) &&\n-\t\t !strcmp(rewritten_ref, head_referent.buf)) {\n-\t\t\t/*\n-\t\t\t * First make sure that HEAD is not already in the\n-\t\t\t * transaction. This check is O(lg N) in the transaction\n-\t\t\t * size, but it happens at most once per transaction.\n-\t\t\t */\n-\t\t\tif (string_list_has_string(&transaction->refnames, \"HEAD\")) {\n-\t\t\t\t/* An entry already existed */\n-\t\t\t\tstrbuf_addf(err,\n-\t\t\t\t\t _(\"multiple updates for 'HEAD' (including one \"\n-\t\t\t\t\t \"via its referent '%s') are not allowed\"),\n-\t\t\t\t\t u->refname);\n-\t\t\t\tret = TRANSACTION_NAME_CONFLICT;\n-\t\t\t\tgoto done;\n-\t\t\t}\n-\n-\t\t\tref_transaction_add_update(\n-\t\t\t\ttransaction, \"HEAD\",\n-\t\t\t\tu->flags | REF_LOG_ONLY | REF_NO_DEREF,\n-\t\t\t\t&u->new_oid, &u->old_oid, NULL, NULL, NULL,\n-\t\t\t\tu->msg);\n-\t\t}\n-\n-\t\tret = reftable_backend_read_ref(be, rewritten_ref,\n-\t\t\t\t\t\t¤t_oid, &referent, &u->type);\n-\t\tif (ret < 0)\n-\t\t\tgoto done;\n-\t\tif (ret > 0 && !ref_update_expects_existing_old_ref(u)) {\n-\t\t\t/*\n-\t\t\t * The reference does not exist, and we either have no\n-\t\t\t * old object ID or expect the reference to not exist.\n-\t\t\t * We can thus skip below safety checks as well as the\n-\t\t\t * symref splitting. But we do want to verify that\n-\t\t\t * there is no conflicting reference here so that we\n-\t\t\t * can output a proper error message instead of failing\n-\t\t\t * at a later point.\n-\t\t\t */\n-\t\t\tstring_list_append(&refnames_to_check, u->refname);\n-\n-\t\t\t/*\n-\t\t\t * There is no need to write the reference deletion\n-\t\t\t * when the reference in question doesn't exist.\n-\t\t\t */\n-\t\t\t if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {\n-\t\t\t\t ret = queue_transaction_update(refs, tx_data, u,\n-\t\t\t\t\t\t\t\t¤t_oid, err);\n-\t\t\t\t if (ret)\n-\t\t\t\t\t goto done;\n-\t\t\t }\n-\n-\t\t\tcontinue;\n-\t\t}\n-\t\tif (ret > 0) {\n-\t\t\t/* The reference does not exist, but we expected it to. */\n-\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n-\t\t\t\t \"unable to resolve reference '%s'\"),\n-\t\t\t\t ref_update_original_update_refname(u), u->refname);\n-\t\t\tret = -1;\n-\t\t\tgoto done;\n-\t\t}\n-\n-\t\tif (u->type & REF_ISSYMREF) {\n-\t\t\t/*\n-\t\t\t * The reftable stack is locked at this point already,\n-\t\t\t * so it is safe to call `refs_resolve_ref_unsafe()`\n-\t\t\t * here without causing races.\n-\t\t\t */\n-\t\t\tconst char *resolved = refs_resolve_ref_unsafe(&refs->base, u->refname, 0,\n-\t\t\t\t\t\t\t\t ¤t_oid, NULL);\n-\n-\t\t\tif (u->flags & REF_NO_DEREF) {\n-\t\t\t\tif (u->flags & REF_HAVE_OLD && !resolved) {\n-\t\t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n-\t\t\t\t\t\t \"error reading reference\"), u->refname);\n-\t\t\t\t\tret = -1;\n-\t\t\t\t\tgoto done;\n-\t\t\t\t}\n-\t\t\t} else {\n-\t\t\t\tstruct ref_update *new_update;\n-\t\t\t\tint new_flags;\n-\n-\t\t\t\tnew_flags = u->flags;\n-\t\t\t\tif (!strcmp(rewritten_ref, \"HEAD\"))\n-\t\t\t\t\tnew_flags |= REF_UPDATE_VIA_HEAD;\n-\n-\t\t\t\tif (string_list_has_string(&transaction->refnames, referent.buf)) {\n-\t\t\t\t\tstrbuf_addf(err,\n-\t\t\t\t\t\t _(\"multiple updates for '%s' (including one \"\n-\t\t\t\t\t\t \"via symref '%s') are not allowed\"),\n-\t\t\t\t\t\t referent.buf, u->refname);\n-\t\t\t\t\tret = TRANSACTION_NAME_CONFLICT;\n-\t\t\t\t\tgoto done;\n-\t\t\t\t}\n-\n-\t\t\t\t/*\n-\t\t\t\t * If we are updating a symref (eg. HEAD), we should also\n-\t\t\t\t * update the branch that the symref points to.\n-\t\t\t\t *\n-\t\t\t\t * This is generic functionality, and would be better\n-\t\t\t\t * done in refs.c, but the current implementation is\n-\t\t\t\t * intertwined with the locking in files-backend.c.\n-\t\t\t\t */\n-\t\t\t\tnew_update = ref_transaction_add_update(\n-\t\t\t\t\ttransaction, referent.buf, new_flags,\n-\t\t\t\t\tu->new_target ? NULL : &u->new_oid,\n-\t\t\t\t\tu->old_target ? NULL : &u->old_oid,\n-\t\t\t\t\tu->new_target, u->old_target,\n-\t\t\t\t\tu->committer_info, u->msg);\n-\n-\t\t\t\tnew_update->parent_update = u;\n-\n-\t\t\t\t/*\n-\t\t\t\t * Change the symbolic ref update to log only. Also, it\n-\t\t\t\t * doesn't need to check its old OID value, as that will be\n-\t\t\t\t * done when new_update is processed.\n-\t\t\t\t */\n-\t\t\t\tu->flags |= REF_LOG_ONLY | REF_NO_DEREF;\n-\t\t\t\tu->flags &= ~REF_HAVE_OLD;\n-\t\t\t}\n-\t\t}\n-\n-\t\t/*\n-\t\t * Verify that the old object matches our expectations. Note\n-\t\t * that the error messages here do not make a lot of sense in\n-\t\t * the context of the reftable backend as we never lock\n-\t\t * individual refs. But the error messages match what the files\n-\t\t * backend returns, which keeps our tests happy.\n-\t\t */\n-\t\tif (u->old_target) {\n-\t\t\tif (!(u->type & REF_ISSYMREF)) {\n-\t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n-\t\t\t\t\t \"expected symref with target '%s': \"\n-\t\t\t\t\t \"but is a regular ref\"),\n-\t\t\t\t\t ref_update_original_update_refname(u),\n-\t\t\t\t\t u->old_target);\n-\t\t\t\tret = -1;\n-\t\t\t\tgoto done;\n-\t\t\t}\n-\n-\t\t\tif (ref_update_check_old_target(referent.buf, u, err)) {\n-\t\t\t\tret = -1;\n-\t\t\t\tgoto done;\n-\t\t\t}\n-\t\t} else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {\n-\t\t\tret = TRANSACTION_NAME_CONFLICT;\n-\t\t\tif (is_null_oid(&u->old_oid)) {\n-\t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n-\t\t\t\t\t\t \"reference already exists\"),\n-\t\t\t\t\t ref_update_original_update_refname(u));\n-\t\t\t\tret = TRANSACTION_CREATE_EXISTS;\n-\t\t\t}\n-\t\t\telse if (is_null_oid(¤t_oid))\n-\t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n-\t\t\t\t\t\t \"reference is missing but expected %s\"),\n-\t\t\t\t\t ref_update_original_update_refname(u),\n-\t\t\t\t\t oid_to_hex(&u->old_oid));\n-\t\t\telse\n-\t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n-\t\t\t\t\t\t \"is at %s but expected %s\"),\n-\t\t\t\t\t ref_update_original_update_refname(u),\n-\t\t\t\t\t oid_to_hex(¤t_oid),\n-\t\t\t\t\t oid_to_hex(&u->old_oid));\n-\t\t\tgoto done;\n-\t\t}\n-\n-\t\t/*\n-\t\t * If all of the following conditions are true:\n-\t\t *\n-\t\t * - We're not about to write a symref.\n-\t\t * - We're not about to write a log-only entry.\n-\t\t * - Old and new object ID are different.\n-\t\t *\n-\t\t * Then we're essentially doing a no-op update that can be\n-\t\t * skipped. This is not only for the sake of efficiency, but\n-\t\t * also skips writing unneeded reflog entries.\n-\t\t */\n-\t\tif ((u->type & REF_ISSYMREF) ||\n-\t\t (u->flags & REF_LOG_ONLY) ||\n-\t\t (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid))) {\n-\t\t\tret = queue_transaction_update(refs, tx_data, u,\n-\t\t\t\t\t\t ¤t_oid, err);\n-\t\t\tif (ret)\n-\t\t\t\tgoto done;\n-\t\t}\n \t}\n \n \tret = refs_verify_refnames_available(ref_store, &refnames_to_check,\n\n-- \n2.48.1\n\n"},{"id":"515181","messageId":"20250327-245-partially-atomic-ref-updates-v5-5-4db2a3e34404@gmail.com","threadId":"62915","inReplyTo":"20250327-245-partially-atomic-ref-updates-v5-0-4db2a3e34404@gmail.com","subject":"[PATCH v5 5/8] refs: introduce enum-based transaction error types","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-03-27T11:13:29Z","receivedAt":"2025-03-27T11:13:37Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Replace preprocessor-defined transaction errors with a strongly-typed\nenum `ref_transaction_error`. This change:\n\n - Improves type safety and function signature clarity.\n - Makes error handling more explicit and discoverable.\n - Maintains existing error cases, while adding new error cases for\n common scenarios.\n\nThis refactoring paves the way for more comprehensive error handling\nwhich we will utilize in the upcoming commits to add batch reference\nupdate support.\n\nSigned-off-by: Karthik Nayak \n---\n builtin/fetch.c | 2 +-\n refs.c | 49 ++++++------\n refs.h | 48 +++++++-----\n refs/files-backend.c | 202 ++++++++++++++++++++++++------------------------\n refs/packed-backend.c | 23 +++---\n refs/refs-internal.h | 5 +-\n refs/reftable-backend.c | 64 +++++++--------\n 7 files changed, 207 insertions(+), 186 deletions(-)\n\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex 95fd0018b9..7615c17faf 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -687,7 +687,7 @@ static int s_update_ref(const char *action,\n \t\tswitch (ref_transaction_commit(our_transaction, &err)) {\n \t\tcase 0:\n \t\t\tbreak;\n-\t\tcase TRANSACTION_NAME_CONFLICT:\n+\t\tcase REF_TRANSACTION_ERROR_NAME_CONFLICT:\n \t\t\tret = STORE_REF_ERROR_DF_CONFLICT;\n \t\t\tgoto out;\n \t\tdefault:\ndiff --git a/refs.c b/refs.c\nindex 61bed9672a..3d0b53d56e 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2271,7 +2271,7 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,\n \t\t\t\t\t REF_NO_DEREF, logmsg, &err))\n \t\t\tgoto error_return;\n \t\tprepret = ref_transaction_prepare(transaction, &err);\n-\t\tif (prepret && prepret != TRANSACTION_CREATE_EXISTS)\n+\t\tif (prepret && prepret != REF_TRANSACTION_ERROR_CREATE_EXISTS)\n \t\t\tgoto error_return;\n \t} else {\n \t\tif (ref_transaction_update(transaction, ref, NULL, NULL,\n@@ -2289,7 +2289,7 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,\n \t\t}\n \t}\n \n-\tif (prepret == TRANSACTION_CREATE_EXISTS)\n+\tif (prepret == REF_TRANSACTION_ERROR_CREATE_EXISTS)\n \t\tgoto cleanup;\n \n \tif (ref_transaction_commit(transaction, &err))\n@@ -2425,7 +2425,7 @@ int ref_transaction_prepare(struct ref_transaction *transaction,\n \n \tstring_list_sort(&transaction->refnames);\n \tif (ref_update_reject_duplicates(&transaction->refnames, err))\n-\t\treturn TRANSACTION_GENERIC_ERROR;\n+\t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \n \tret = refs->be->transaction_prepare(refs, transaction, err);\n \tif (ret)\n@@ -2497,19 +2497,19 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \treturn ret;\n }\n \n-int refs_verify_refnames_available(struct ref_store *refs,\n-\t\t\t\t const struct string_list *refnames,\n-\t\t\t\t const struct string_list *extras,\n-\t\t\t\t const struct string_list *skip,\n-\t\t\t\t unsigned int initial_transaction,\n-\t\t\t\t struct strbuf *err)\n+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,\n+\t\t\t\t\t const struct string_list *refnames,\n+\t\t\t\t\t const struct string_list *extras,\n+\t\t\t\t\t const struct string_list *skip,\n+\t\t\t\t\t unsigned int initial_transaction,\n+\t\t\t\t\t struct strbuf *err)\n {\n \tstruct strbuf dirname = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct string_list_item *item;\n \tstruct ref_iterator *iter = NULL;\n \tstruct strset dirnames;\n-\tint ret = -1;\n+\tint ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \n \t/*\n \t * For the sake of comments in this function, suppose that\n@@ -2625,12 +2625,13 @@ int refs_verify_refnames_available(struct ref_store *refs,\n \treturn ret;\n }\n \n-int refs_verify_refname_available(struct ref_store *refs,\n-\t\t\t\t const char *refname,\n-\t\t\t\t const struct string_list *extras,\n-\t\t\t\t const struct string_list *skip,\n-\t\t\t\t unsigned int initial_transaction,\n-\t\t\t\t struct strbuf *err)\n+enum ref_transaction_error refs_verify_refname_available(\n+\tstruct ref_store *refs,\n+\tconst char *refname,\n+\tconst struct string_list *extras,\n+\tconst struct string_list *skip,\n+\tunsigned int initial_transaction,\n+\tstruct strbuf *err)\n {\n \tstruct string_list_item item = { .string = (char *) refname };\n \tstruct string_list refnames = {\n@@ -2818,8 +2819,9 @@ int ref_update_has_null_new_value(struct ref_update *update)\n \treturn !update->new_target && is_null_oid(&update->new_oid);\n }\n \n-int ref_update_check_old_target(const char *referent, struct ref_update *update,\n-\t\t\t\tstruct strbuf *err)\n+enum ref_transaction_error ref_update_check_old_target(const char *referent,\n+\t\t\t\t\t\t struct ref_update *update,\n+\t\t\t\t\t\t struct strbuf *err)\n {\n \tif (!update->old_target)\n \t\tBUG(\"called without old_target set\");\n@@ -2827,17 +2829,18 @@ int ref_update_check_old_target(const char *referent, struct ref_update *update,\n \tif (!strcmp(referent, update->old_target))\n \t\treturn 0;\n \n-\tif (!strcmp(referent, \"\"))\n+\tif (!strcmp(referent, \"\")) {\n \t\tstrbuf_addf(err, \"verifying symref target: '%s': \"\n \t\t\t \"reference is missing but expected %s\",\n \t\t\t ref_update_original_update_refname(update),\n \t\t\t update->old_target);\n-\telse\n-\t\tstrbuf_addf(err, \"verifying symref target: '%s': \"\n-\t\t\t \"is at %s but expected %s\",\n+\t\treturn REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n+\t}\n+\n+\tstrbuf_addf(err, \"verifying symref target: '%s': is at %s but expected %s\",\n \t\t\t ref_update_original_update_refname(update),\n \t\t\t referent, update->old_target);\n-\treturn -1;\n+\treturn REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;\n }\n \n struct migration_data {\ndiff --git a/refs.h b/refs.h\nindex 240e2d8537..f009cdae7d 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -16,6 +16,23 @@ struct worktree;\n enum ref_storage_format ref_storage_format_by_name(const char *name);\n const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format);\n \n+enum ref_transaction_error {\n+\t/* Default error code */\n+\tREF_TRANSACTION_ERROR_GENERIC = -1,\n+\t/* Ref name conflict like A vs A/B */\n+\tREF_TRANSACTION_ERROR_NAME_CONFLICT = -2,\n+\t/* Ref to be created already exists */\n+\tREF_TRANSACTION_ERROR_CREATE_EXISTS = -3,\n+\t/* ref expected but doesn't exist */\n+\tREF_TRANSACTION_ERROR_NONEXISTENT_REF = -4,\n+\t/* Provided old_oid or old_target of reference doesn't match actual */\n+\tREF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE = -5,\n+\t/* Provided new_oid or new_target is invalid */\n+\tREF_TRANSACTION_ERROR_INVALID_NEW_VALUE = -6,\n+\t/* Expected ref to be symref, but is a regular ref */\n+\tREF_TRANSACTION_ERROR_EXPECTED_SYMREF = -7,\n+};\n+\n /*\n * Resolve a reference, recursively following symbolic references.\n *\n@@ -117,24 +134,24 @@ int refs_read_symbolic_ref(struct ref_store *ref_store, const char *refname,\n *\n * extras and skip must be sorted.\n */\n-int refs_verify_refname_available(struct ref_store *refs,\n-\t\t\t\t const char *refname,\n-\t\t\t\t const struct string_list *extras,\n-\t\t\t\t const struct string_list *skip,\n-\t\t\t\t unsigned int initial_transaction,\n-\t\t\t\t struct strbuf *err);\n+enum ref_transaction_error refs_verify_refname_available(struct ref_store *refs,\n+\t\t\t\t\t\t const char *refname,\n+\t\t\t\t\t\t const struct string_list *extras,\n+\t\t\t\t\t\t const struct string_list *skip,\n+\t\t\t\t\t\t unsigned int initial_transaction,\n+\t\t\t\t\t\t struct strbuf *err);\n \n /*\n * Same as `refs_verify_refname_available()`, but checking for a list of\n * refnames instead of only a single item. This is more efficient in the case\n * where one needs to check multiple refnames.\n */\n-int refs_verify_refnames_available(struct ref_store *refs,\n-\t\t\t\t const struct string_list *refnames,\n-\t\t\t\t const struct string_list *extras,\n-\t\t\t\t const struct string_list *skip,\n-\t\t\t\t unsigned int initial_transaction,\n-\t\t\t\t struct strbuf *err);\n+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,\n+\t\t\t\t\t const struct string_list *refnames,\n+\t\t\t\t\t const struct string_list *extras,\n+\t\t\t\t\t const struct string_list *skip,\n+\t\t\t\t\t unsigned int initial_transaction,\n+\t\t\t\t\t struct strbuf *err);\n \n int refs_ref_exists(struct ref_store *refs, const char *refname);\n \n@@ -830,13 +847,6 @@ int ref_transaction_verify(struct ref_transaction *transaction,\n \t\t\t unsigned int flags,\n \t\t\t struct strbuf *err);\n \n-/* Naming conflict (for example, the ref names A and A/B conflict). */\n-#define TRANSACTION_NAME_CONFLICT -1\n-/* When only creation was requested, but the ref already exists. */\n-#define TRANSACTION_CREATE_EXISTS -2\n-/* All other errors. */\n-#define TRANSACTION_GENERIC_ERROR -3\n-\n /*\n * Perform the preparatory stages of committing `transaction`. Acquire\n * any needed locks, check preconditions, etc.; basically, do as much\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex ea023a59fc..4f27f7652c 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -663,7 +663,7 @@ static void unlock_ref(struct ref_lock *lock)\n * broken, lock the reference anyway but clear old_oid.\n *\n * Return 0 on success. On failure, write an error message to err and\n- * return TRANSACTION_NAME_CONFLICT or TRANSACTION_GENERIC_ERROR.\n+ * return REF_TRANSACTION_ERROR_NAME_CONFLICT or REF_TRANSACTION_ERROR_GENERIC.\n *\n * Implementation note: This function is basically\n *\n@@ -676,19 +676,20 @@ static void unlock_ref(struct ref_lock *lock)\n * avoided, namely if we were successfully able to read the ref\n * - Generate informative error messages in the case of failure\n */\n-static int lock_raw_ref(struct files_ref_store *refs,\n-\t\t\tconst char *refname, int mustexist,\n-\t\t\tstruct string_list *refnames_to_check,\n-\t\t\tconst struct string_list *extras,\n-\t\t\tstruct ref_lock **lock_p,\n-\t\t\tstruct strbuf *referent,\n-\t\t\tunsigned int *type,\n-\t\t\tstruct strbuf *err)\n-{\n+static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int mustexist,\n+\t\t\t\t\t struct string_list *refnames_to_check,\n+\t\t\t\t\t const struct string_list *extras,\n+\t\t\t\t\t struct ref_lock **lock_p,\n+\t\t\t\t\t struct strbuf *referent,\n+\t\t\t\t\t unsigned int *type,\n+\t\t\t\t\t struct strbuf *err)\n+{\n+\tenum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;\n \tstruct ref_lock *lock;\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tint attempts_remaining = 3;\n-\tint ret = TRANSACTION_GENERIC_ERROR;\n \tint failure_errno;\n \n \tassert(err);\n@@ -728,13 +729,14 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\tstrbuf_reset(err);\n \t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s'\",\n \t\t\t\t\t refname);\n+\t\t\t\tret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n \t\t\t} else {\n \t\t\t\t/*\n \t\t\t\t * The error message set by\n \t\t\t\t * refs_verify_refname_available() is\n \t\t\t\t * OK.\n \t\t\t\t */\n-\t\t\t\tret = TRANSACTION_NAME_CONFLICT;\n+\t\t\t\tret = REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t\t\t}\n \t\t} else {\n \t\t\t/*\n@@ -788,6 +790,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t/* Garden variety missing reference. */\n \t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s'\",\n \t\t\t\t\t refname);\n+\t\t\t\tret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n \t\t\t\tgoto error_return;\n \t\t\t} else {\n \t\t\t\t/*\n@@ -820,6 +823,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t/* Garden variety missing reference. */\n \t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s'\",\n \t\t\t\t\t refname);\n+\t\t\t\tret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n \t\t\t\tgoto error_return;\n \t\t\t} else if (remove_dir_recursively(&ref_file,\n \t\t\t\t\t\t\t REMOVE_DIR_EMPTY_ONLY)) {\n@@ -830,7 +834,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t\t * The error message set by\n \t\t\t\t\t * verify_refname_available() is OK.\n \t\t\t\t\t */\n-\t\t\t\t\tret = TRANSACTION_NAME_CONFLICT;\n+\t\t\t\t\tret = REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t\t\t\t\tgoto error_return;\n \t\t\t\t} else {\n \t\t\t\t\t/*\n@@ -1517,10 +1521,11 @@ static int rename_tmp_log(struct files_ref_store *refs, const char *newrefname)\n \treturn ret;\n }\n \n-static int write_ref_to_lockfile(struct files_ref_store *refs,\n-\t\t\t\t struct ref_lock *lock,\n-\t\t\t\t const struct object_id *oid,\n-\t\t\t\t int skip_oid_verification, struct strbuf *err);\n+static enum ref_transaction_error write_ref_to_lockfile(struct files_ref_store *refs,\n+\t\t\t\t\t\t\tstruct ref_lock *lock,\n+\t\t\t\t\t\t\tconst struct object_id *oid,\n+\t\t\t\t\t\t\tint skip_oid_verification,\n+\t\t\t\t\t\t\tstruct strbuf *err);\n static int commit_ref_update(struct files_ref_store *refs,\n \t\t\t struct ref_lock *lock,\n \t\t\t const struct object_id *oid, const char *logmsg,\n@@ -1926,10 +1931,11 @@ static int files_log_ref_write(struct files_ref_store *refs,\n * Write oid into the open lockfile, then close the lockfile. On\n * errors, rollback the lockfile, fill in *err and return -1.\n */\n-static int write_ref_to_lockfile(struct files_ref_store *refs,\n-\t\t\t\t struct ref_lock *lock,\n-\t\t\t\t const struct object_id *oid,\n-\t\t\t\t int skip_oid_verification, struct strbuf *err)\n+static enum ref_transaction_error write_ref_to_lockfile(struct files_ref_store *refs,\n+\t\t\t\t\t\t\tstruct ref_lock *lock,\n+\t\t\t\t\t\t\tconst struct object_id *oid,\n+\t\t\t\t\t\t\tint skip_oid_verification,\n+\t\t\t\t\t\t\tstruct strbuf *err)\n {\n \tstatic char term = '\\n';\n \tstruct object *o;\n@@ -1943,7 +1949,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,\n \t\t\t\t\"trying to write ref '%s' with nonexistent object %s\",\n \t\t\t\tlock->ref_name, oid_to_hex(oid));\n \t\t\tunlock_ref(lock);\n-\t\t\treturn -1;\n+\t\t\treturn REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;\n \t\t}\n \t\tif (o->type != OBJ_COMMIT && is_branch(lock->ref_name)) {\n \t\t\tstrbuf_addf(\n@@ -1951,7 +1957,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,\n \t\t\t\t\"trying to write non-commit object %s to branch '%s'\",\n \t\t\t\toid_to_hex(oid), lock->ref_name);\n \t\t\tunlock_ref(lock);\n-\t\t\treturn -1;\n+\t\t\treturn REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;\n \t\t}\n \t}\n \tfd = get_lock_file_fd(&lock->lk);\n@@ -1962,7 +1968,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,\n \t\tstrbuf_addf(err,\n \t\t\t \"couldn't write '%s'\", get_lock_file_path(&lock->lk));\n \t\tunlock_ref(lock);\n-\t\treturn -1;\n+\t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \t}\n \treturn 0;\n }\n@@ -2376,9 +2382,10 @@ static struct ref_iterator *files_reflog_iterator_begin(struct ref_store *ref_st\n * If update is a direct update of head_ref (the reference pointed to\n * by HEAD), then add an extra REF_LOG_ONLY update for HEAD.\n */\n-static int split_head_update(struct ref_update *update,\n-\t\t\t struct ref_transaction *transaction,\n-\t\t\t const char *head_ref, struct strbuf *err)\n+static enum ref_transaction_error split_head_update(struct ref_update *update,\n+\t\t\t\t\t\t struct ref_transaction *transaction,\n+\t\t\t\t\t\t const char *head_ref,\n+\t\t\t\t\t\t struct strbuf *err)\n {\n \tstruct ref_update *new_update;\n \n@@ -2402,7 +2409,7 @@ static int split_head_update(struct ref_update *update,\n \t\t\t \"multiple updates for 'HEAD' (including one \"\n \t\t\t \"via its referent '%s') are not allowed\",\n \t\t\t update->refname);\n-\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t\treturn REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t}\n \n \tnew_update = ref_transaction_add_update(\n@@ -2430,10 +2437,10 @@ static int split_head_update(struct ref_update *update,\n * Note that the new update will itself be subject to splitting when\n * the iteration gets to it.\n */\n-static int split_symref_update(struct ref_update *update,\n-\t\t\t const char *referent,\n-\t\t\t struct ref_transaction *transaction,\n-\t\t\t struct strbuf *err)\n+static enum ref_transaction_error split_symref_update(struct ref_update *update,\n+\t\t\t\t\t\t const char *referent,\n+\t\t\t\t\t\t struct ref_transaction *transaction,\n+\t\t\t\t\t\t struct strbuf *err)\n {\n \tstruct ref_update *new_update;\n \tunsigned int new_flags;\n@@ -2450,7 +2457,7 @@ static int split_symref_update(struct ref_update *update,\n \t\t\t \"multiple updates for '%s' (including one \"\n \t\t\t \"via symref '%s') are not allowed\",\n \t\t\t referent, update->refname);\n-\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t\treturn REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t}\n \n \tnew_flags = update->flags;\n@@ -2491,11 +2498,10 @@ static int split_symref_update(struct ref_update *update,\n * everything is OK, return 0; otherwise, write an error message to\n * err and return -1.\n */\n-static int check_old_oid(struct ref_update *update, struct object_id *oid,\n-\t\t\t struct strbuf *err)\n+static enum ref_transaction_error check_old_oid(struct ref_update *update,\n+\t\t\t\t\t\tstruct object_id *oid,\n+\t\t\t\t\t\tstruct strbuf *err)\n {\n-\tint ret = TRANSACTION_GENERIC_ERROR;\n-\n \tif (!(update->flags & REF_HAVE_OLD) ||\n \t\t oideq(oid, &update->old_oid))\n \t\treturn 0;\n@@ -2504,21 +2510,20 @@ static int check_old_oid(struct ref_update *update, struct object_id *oid,\n \t\tstrbuf_addf(err, \"cannot lock ref '%s': \"\n \t\t\t \"reference already exists\",\n \t\t\t ref_update_original_update_refname(update));\n-\t\tret = TRANSACTION_CREATE_EXISTS;\n-\t}\n-\telse if (is_null_oid(oid))\n+\t\treturn REF_TRANSACTION_ERROR_CREATE_EXISTS;\n+\t} else if (is_null_oid(oid)) {\n \t\tstrbuf_addf(err, \"cannot lock ref '%s': \"\n \t\t\t \"reference is missing but expected %s\",\n \t\t\t ref_update_original_update_refname(update),\n \t\t\t oid_to_hex(&update->old_oid));\n-\telse\n-\t\tstrbuf_addf(err, \"cannot lock ref '%s': \"\n-\t\t\t \"is at %s but expected %s\",\n-\t\t\t ref_update_original_update_refname(update),\n-\t\t\t oid_to_hex(oid),\n-\t\t\t oid_to_hex(&update->old_oid));\n+\t\treturn REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n+\t}\n \n-\treturn ret;\n+\tstrbuf_addf(err, \"cannot lock ref '%s': is at %s but expected %s\",\n+\t\t ref_update_original_update_refname(update), oid_to_hex(oid),\n+\t\t oid_to_hex(&update->old_oid));\n+\n+\treturn REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;\n }\n \n struct files_transaction_backend_data {\n@@ -2540,17 +2545,17 @@ struct files_transaction_backend_data {\n * - If it is an update of head_ref, add a corresponding REF_LOG_ONLY\n * update of HEAD.\n */\n-static int lock_ref_for_update(struct files_ref_store *refs,\n-\t\t\t struct ref_update *update,\n-\t\t\t struct ref_transaction *transaction,\n-\t\t\t const char *head_ref,\n-\t\t\t struct string_list *refnames_to_check,\n-\t\t\t struct strbuf *err)\n+static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *refs,\n+\t\t\t\t\t\t struct ref_update *update,\n+\t\t\t\t\t\t struct ref_transaction *transaction,\n+\t\t\t\t\t\t const char *head_ref,\n+\t\t\t\t\t\t struct string_list *refnames_to_check,\n+\t\t\t\t\t\t struct strbuf *err)\n {\n \tstruct strbuf referent = STRBUF_INIT;\n \tint mustexist = ref_update_expects_existing_old_ref(update);\n \tstruct files_transaction_backend_data *backend_data;\n-\tint ret = 0;\n+\tenum ref_transaction_error ret = 0;\n \tstruct ref_lock *lock;\n \n \tfiles_assert_main_repository(refs, \"lock_ref_for_update\");\n@@ -2602,22 +2607,17 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\t\t\t\tstrbuf_addf(err, \"cannot lock ref '%s': \"\n \t\t\t\t\t\t \"error reading reference\",\n \t\t\t\t\t\t ref_update_original_update_refname(update));\n-\t\t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\t\t\tgoto out;\n \t\t\t\t}\n \t\t\t}\n \n-\t\t\tif (update->old_target) {\n-\t\t\t\tif (ref_update_check_old_target(referent.buf, update, err)) {\n-\t\t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n-\t\t\t\t\tgoto out;\n-\t\t\t\t}\n-\t\t\t} else {\n+\t\t\tif (update->old_target)\n+\t\t\t\tret = ref_update_check_old_target(referent.buf, update, err);\n+\t\t\telse\n \t\t\t\tret = check_old_oid(update, &lock->old_oid, err);\n-\t\t\t\tif (ret) {\n-\t\t\t\t\tgoto out;\n-\t\t\t\t}\n-\t\t\t}\n+\t\t\tif (ret)\n+\t\t\t\tgoto out;\n \t\t} else {\n \t\t\t/*\n \t\t\t * Create a new update for the reference this\n@@ -2644,7 +2644,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\t\t\t\t \"but is a regular ref\"),\n \t\t\t\t ref_update_original_update_refname(update),\n \t\t\t\t update->old_target);\n-\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\tret = REF_TRANSACTION_ERROR_EXPECTED_SYMREF;\n \t\t\tgoto out;\n \t\t} else {\n \t\t\tret = check_old_oid(update, &lock->old_oid, err);\n@@ -2668,14 +2668,14 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \n \tif (update->new_target && !(update->flags & REF_LOG_ONLY)) {\n \t\tif (create_symref_lock(lock, update->new_target, err)) {\n-\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\tgoto out;\n \t\t}\n \n \t\tif (close_ref_gently(lock)) {\n \t\t\tstrbuf_addf(err, \"couldn't close '%s.lock'\",\n \t\t\t\t update->refname);\n-\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\tgoto out;\n \t\t}\n \n@@ -2693,25 +2693,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\t\t * The reference already has the desired\n \t\t\t * value, so we don't need to write it.\n \t\t\t */\n-\t\t} else if (write_ref_to_lockfile(\n-\t\t\t\t refs, lock, &update->new_oid,\n-\t\t\t\t update->flags & REF_SKIP_OID_VERIFICATION,\n-\t\t\t\t err)) {\n-\t\t\tchar *write_err = strbuf_detach(err, NULL);\n-\n-\t\t\t/*\n-\t\t\t * The lock was freed upon failure of\n-\t\t\t * write_ref_to_lockfile():\n-\t\t\t */\n-\t\t\tupdate->backend_data = NULL;\n-\t\t\tstrbuf_addf(err,\n-\t\t\t\t \"cannot update ref '%s': %s\",\n-\t\t\t\t update->refname, write_err);\n-\t\t\tfree(write_err);\n-\t\t\tret = TRANSACTION_GENERIC_ERROR;\n-\t\t\tgoto out;\n \t\t} else {\n-\t\t\tupdate->flags |= REF_NEEDS_COMMIT;\n+\t\t\tret = write_ref_to_lockfile(\n+\t\t\t\trefs, lock, &update->new_oid,\n+\t\t\t\tupdate->flags & REF_SKIP_OID_VERIFICATION,\n+\t\t\t\terr);\n+\t\t\tif (ret) {\n+\t\t\t\tchar *write_err = strbuf_detach(err, NULL);\n+\n+\t\t\t\t/*\n+\t\t\t\t * The lock was freed upon failure of\n+\t\t\t\t * write_ref_to_lockfile():\n+\t\t\t\t */\n+\t\t\t\tupdate->backend_data = NULL;\n+\t\t\t\tstrbuf_addf(err,\n+\t\t\t\t\t \"cannot update ref '%s': %s\",\n+\t\t\t\t\t update->refname, write_err);\n+\t\t\t\tfree(write_err);\n+\t\t\t\tgoto out;\n+\t\t\t} else {\n+\t\t\t\tupdate->flags |= REF_NEEDS_COMMIT;\n+\t\t\t}\n \t\t}\n \t}\n \tif (!(update->flags & REF_NEEDS_COMMIT)) {\n@@ -2723,7 +2725,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\tif (close_ref_gently(lock)) {\n \t\t\tstrbuf_addf(err, \"couldn't close '%s.lock'\",\n \t\t\t\t update->refname);\n-\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\tgoto out;\n \t\t}\n \t}\n@@ -2865,7 +2867,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\t\t\trefs->packed_ref_store,\n \t\t\t\t\t\ttransaction->flags, err);\n \t\t\t\tif (!packed_transaction) {\n-\t\t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\t\t\tgoto cleanup;\n \t\t\t\t}\n \n@@ -2897,13 +2899,13 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t */\n \tif (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,\n \t\t\t\t\t &transaction->refnames, NULL, 0, err)) {\n-\t\tret = TRANSACTION_NAME_CONFLICT;\n+\t\tret = REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t\tgoto cleanup;\n \t}\n \n \tif (packed_transaction) {\n \t\tif (packed_refs_lock(refs->packed_ref_store, 0, err)) {\n-\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\tgoto cleanup;\n \t\t}\n \t\tbackend_data->packed_refs_locked = 1;\n@@ -2934,7 +2936,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t\t\t */\n \t\t\tbackend_data->packed_transaction = NULL;\n \t\t\tif (ref_transaction_abort(packed_transaction, err)) {\n-\t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\t\tgoto cleanup;\n \t\t\t}\n \t\t}\n@@ -3035,7 +3037,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \tpacked_transaction = ref_store_transaction_begin(refs->packed_ref_store,\n \t\t\t\t\t\t\t transaction->flags, err);\n \tif (!packed_transaction) {\n-\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\tgoto cleanup;\n \t}\n \n@@ -3058,7 +3060,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \t\t\tif (!loose_transaction) {\n \t\t\t\tloose_transaction = ref_store_transaction_begin(&refs->base, 0, err);\n \t\t\t\tif (!loose_transaction) {\n-\t\t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\t\t\tgoto cleanup;\n \t\t\t\t}\n \t\t\t}\n@@ -3083,19 +3085,19 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \t}\n \n \tif (packed_refs_lock(refs->packed_ref_store, 0, err)) {\n-\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\tgoto cleanup;\n \t}\n \n \tif (refs_verify_refnames_available(&refs->base, &refnames_to_check,\n \t\t\t\t\t &affected_refnames, NULL, 1, err)) {\n \t\tpacked_refs_unlock(refs->packed_ref_store);\n-\t\tret = TRANSACTION_NAME_CONFLICT;\n+\t\tret = REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t\tgoto cleanup;\n \t}\n \n \tif (ref_transaction_commit(packed_transaction, err)) {\n-\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\tgoto cleanup;\n \t}\n \tpacked_refs_unlock(refs->packed_ref_store);\n@@ -3103,7 +3105,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \tif (loose_transaction) {\n \t\tif (ref_transaction_prepare(loose_transaction, err) ||\n \t\t ref_transaction_commit(loose_transaction, err)) {\n-\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\tgoto cleanup;\n \t\t}\n \t}\n@@ -3152,7 +3154,7 @@ static int files_transaction_finish(struct ref_store *ref_store,\n \t\tif (update->flags & REF_NEEDS_COMMIT ||\n \t\t update->flags & REF_LOG_ONLY) {\n \t\t\tif (parse_and_write_reflog(refs, update, lock, err)) {\n-\t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\t\tgoto cleanup;\n \t\t\t}\n \t\t}\n@@ -3171,7 +3173,7 @@ static int files_transaction_finish(struct ref_store *ref_store,\n \t\t\t\tstrbuf_addf(err, \"couldn't set '%s'\", lock->ref_name);\n \t\t\t\tunlock_ref(lock);\n \t\t\t\tupdate->backend_data = NULL;\n-\t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\t\tgoto cleanup;\n \t\t\t}\n \t\t}\n@@ -3227,7 +3229,7 @@ static int files_transaction_finish(struct ref_store *ref_store,\n \t\t\t\tstrbuf_reset(&sb);\n \t\t\t\tfiles_ref_path(refs, &sb, lock->ref_name);\n \t\t\t\tif (unlink_or_msg(sb.buf, err)) {\n-\t\t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\t\t\tgoto cleanup;\n \t\t\t\t}\n \t\t\t}\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 19220d2e99..d90bd815a3 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1326,10 +1326,11 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,\n * The packfile must be locked before calling this function and will\n * remain locked when it is done.\n */\n-static int write_with_updates(struct packed_ref_store *refs,\n-\t\t\t struct string_list *updates,\n-\t\t\t struct strbuf *err)\n+static enum ref_transaction_error write_with_updates(struct packed_ref_store *refs,\n+\t\t\t\t\t\t struct string_list *updates,\n+\t\t\t\t\t\t struct strbuf *err)\n {\n+\tenum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;\n \tstruct ref_iterator *iter = NULL;\n \tsize_t i;\n \tint ok;\n@@ -1353,7 +1354,7 @@ static int write_with_updates(struct packed_ref_store *refs,\n \t\tstrbuf_addf(err, \"unable to create file %s: %s\",\n \t\t\t sb.buf, strerror(errno));\n \t\tstrbuf_release(&sb);\n-\t\treturn -1;\n+\t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \t}\n \tstrbuf_release(&sb);\n \n@@ -1409,6 +1410,7 @@ static int write_with_updates(struct packed_ref_store *refs,\n \t\t\t\t\tstrbuf_addf(err, \"cannot update ref '%s': \"\n \t\t\t\t\t\t \"reference already exists\",\n \t\t\t\t\t\t update->refname);\n+\t\t\t\t\tret = REF_TRANSACTION_ERROR_CREATE_EXISTS;\n \t\t\t\t\tgoto error;\n \t\t\t\t} else if (!oideq(&update->old_oid, iter->oid)) {\n \t\t\t\t\tstrbuf_addf(err, \"cannot update ref '%s': \"\n@@ -1416,6 +1418,7 @@ static int write_with_updates(struct packed_ref_store *refs,\n \t\t\t\t\t\t update->refname,\n \t\t\t\t\t\t oid_to_hex(iter->oid),\n \t\t\t\t\t\t oid_to_hex(&update->old_oid));\n+\t\t\t\t\tret = REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;\n \t\t\t\t\tgoto error;\n \t\t\t\t}\n \t\t\t}\n@@ -1452,6 +1455,7 @@ static int write_with_updates(struct packed_ref_store *refs,\n \t\t\t\t\t \"reference is missing but expected %s\",\n \t\t\t\t\t update->refname,\n \t\t\t\t\t oid_to_hex(&update->old_oid));\n+\t\t\t\tret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n \t\t\t\tgoto error;\n \t\t\t}\n \t\t}\n@@ -1509,7 +1513,7 @@ static int write_with_updates(struct packed_ref_store *refs,\n \t\t\t strerror(errno));\n \t\tstrbuf_release(&sb);\n \t\tdelete_tempfile(&refs->tempfile);\n-\t\treturn -1;\n+\t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \t}\n \n \treturn 0;\n@@ -1521,7 +1525,7 @@ static int write_with_updates(struct packed_ref_store *refs,\n error:\n \tref_iterator_free(iter);\n \tdelete_tempfile(&refs->tempfile);\n-\treturn -1;\n+\treturn ret;\n }\n \n int is_packed_transaction_needed(struct ref_store *ref_store,\n@@ -1654,7 +1658,7 @@ static int packed_transaction_prepare(struct ref_store *ref_store,\n \t\t\tREF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,\n \t\t\t\"ref_transaction_prepare\");\n \tstruct packed_transaction_backend_data *data;\n-\tint ret = TRANSACTION_GENERIC_ERROR;\n+\tenum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;\n \n \t/*\n \t * Note that we *don't* skip transactions with zero updates,\n@@ -1675,7 +1679,8 @@ static int packed_transaction_prepare(struct ref_store *ref_store,\n \t\tdata->own_lock = 1;\n \t}\n \n-\tif (write_with_updates(refs, &transaction->refnames, err))\n+\tret = write_with_updates(refs, &transaction->refnames, err);\n+\tif (ret)\n \t\tgoto failure;\n \n \ttransaction->state = REF_TRANSACTION_PREPARED;\n@@ -1707,7 +1712,7 @@ static int packed_transaction_finish(struct ref_store *ref_store,\n \t\t\tref_store,\n \t\t\tREF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,\n \t\t\t\"ref_transaction_finish\");\n-\tint ret = TRANSACTION_GENERIC_ERROR;\n+\tint ret = REF_TRANSACTION_ERROR_GENERIC;\n \tchar *packed_refs_path;\n \n \tclear_snapshot(refs);\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 6d3770d0cc..3f1d19abd9 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -770,8 +770,9 @@ int ref_update_has_null_new_value(struct ref_update *update);\n * If everything is OK, return 0; otherwise, write an error message to\n * err and return -1.\n */\n-int ref_update_check_old_target(const char *referent, struct ref_update *update,\n-\t\t\t\tstruct strbuf *err);\n+enum ref_transaction_error ref_update_check_old_target(const char *referent,\n+\t\t\t\t\t\t struct ref_update *update,\n+\t\t\t\t\t\t struct strbuf *err);\n \n /*\n * Check if the ref must exist, this means that the old_oid or\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex 786df11a03..bd6b042103 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -1069,20 +1069,20 @@ static int queue_transaction_update(struct reftable_ref_store *refs,\n \treturn 0;\n }\n \n-static int prepare_single_update(struct reftable_ref_store *refs,\n-\t\t\t\t struct reftable_transaction_data *tx_data,\n-\t\t\t\t struct ref_transaction *transaction,\n-\t\t\t\t struct reftable_backend *be,\n-\t\t\t\t struct ref_update *u,\n-\t\t\t\t struct string_list *refnames_to_check,\n-\t\t\t\t unsigned int head_type,\n-\t\t\t\t struct strbuf *head_referent,\n-\t\t\t\t struct strbuf *referent,\n-\t\t\t\t struct strbuf *err)\n+static enum ref_transaction_error prepare_single_update(struct reftable_ref_store *refs,\n+\t\t\t\t\t\t\tstruct reftable_transaction_data *tx_data,\n+\t\t\t\t\t\t\tstruct ref_transaction *transaction,\n+\t\t\t\t\t\t\tstruct reftable_backend *be,\n+\t\t\t\t\t\t\tstruct ref_update *u,\n+\t\t\t\t\t\t\tstruct string_list *refnames_to_check,\n+\t\t\t\t\t\t\tunsigned int head_type,\n+\t\t\t\t\t\t\tstruct strbuf *head_referent,\n+\t\t\t\t\t\t\tstruct strbuf *referent,\n+\t\t\t\t\t\t\tstruct strbuf *err)\n {\n+\tenum ref_transaction_error ret = 0;\n \tstruct object_id current_oid = {0};\n \tconst char *rewritten_ref;\n-\tint ret = 0;\n \n \t/*\n \t * There is no need to reload the respective backends here as\n@@ -1093,7 +1093,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \t */\n \tret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);\n \tif (ret)\n-\t\treturn ret;\n+\t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \n \t/* Verify that the new object ID is valid. */\n \tif ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&\n@@ -1104,13 +1104,13 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \t\t\tstrbuf_addf(err,\n \t\t\t\t _(\"trying to write ref '%s' with nonexistent object %s\"),\n \t\t\t\t u->refname, oid_to_hex(&u->new_oid));\n-\t\t\treturn -1;\n+\t\t\treturn REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;\n \t\t}\n \n \t\tif (o->type != OBJ_COMMIT && is_branch(u->refname)) {\n \t\t\tstrbuf_addf(err, _(\"trying to write non-commit object %s to branch '%s'\"),\n \t\t\t\t oid_to_hex(&u->new_oid), u->refname);\n-\t\t\treturn -1;\n+\t\t\treturn REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;\n \t\t}\n \t}\n \n@@ -1134,7 +1134,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \t\t\t\t _(\"multiple updates for 'HEAD' (including one \"\n \t\t\t\t \"via its referent '%s') are not allowed\"),\n \t\t\t\t u->refname);\n-\t\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t\t\treturn REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t\t}\n \n \t\tref_transaction_add_update(\n@@ -1147,7 +1147,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \tret = reftable_backend_read_ref(be, rewritten_ref,\n \t\t\t\t\t¤t_oid, referent, &u->type);\n \tif (ret < 0)\n-\t\treturn ret;\n+\t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \tif (ret > 0 && !ref_update_expects_existing_old_ref(u)) {\n \t\t/*\n \t\t * The reference does not exist, and we either have no\n@@ -1168,7 +1168,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \t\t\tret = queue_transaction_update(refs, tx_data, u,\n \t\t\t\t\t\t ¤t_oid, err);\n \t\t\tif (ret)\n-\t\t\t\treturn ret;\n+\t\t\t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \t\t}\n \n \t\treturn 0;\n@@ -1180,7 +1180,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \n \t\t\t\t \"unable to resolve reference '%s'\"),\n \t\t\t ref_update_original_update_refname(u), u->refname);\n-\t\treturn -1;\n+\t\treturn REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n \t}\n \n \tif (u->type & REF_ISSYMREF) {\n@@ -1196,7 +1196,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \t\t\tif (u->flags & REF_HAVE_OLD && !resolved) {\n \t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n \t\t\t\t\t\t \"error reading reference\"), u->refname);\n-\t\t\t\treturn -1;\n+\t\t\t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \t\t\t}\n \t\t} else {\n \t\t\tstruct ref_update *new_update;\n@@ -1211,7 +1211,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \t\t\t\t\t _(\"multiple updates for '%s' (including one \"\n \t\t\t\t\t \"via symref '%s') are not allowed\"),\n \t\t\t\t\t referent->buf, u->refname);\n-\t\t\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t\t\t\treturn REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t\t\t}\n \n \t\t\t/*\n@@ -1255,31 +1255,32 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \t\t\t\t\t \"but is a regular ref\"),\n \t\t\t\t ref_update_original_update_refname(u),\n \t\t\t\t u->old_target);\n-\t\t\treturn -1;\n+\t\t\treturn REF_TRANSACTION_ERROR_EXPECTED_SYMREF;\n \t\t}\n \n-\t\tif (ref_update_check_old_target(referent->buf, u, err)) {\n-\t\t\treturn -1;\n-\t\t}\n+\t\tret = ref_update_check_old_target(referent->buf, u, err);\n+\t\tif (ret)\n+\t\t\treturn ret;\n \t} else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {\n \t\tif (is_null_oid(&u->old_oid)) {\n \t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n \t\t\t\t\t \"reference already exists\"),\n \t\t\t\t ref_update_original_update_refname(u));\n-\t\t\treturn TRANSACTION_CREATE_EXISTS;\n-\t\t}\n-\t\telse if (is_null_oid(¤t_oid))\n+\t\t\treturn REF_TRANSACTION_ERROR_CREATE_EXISTS;\n+\t\t} else if (is_null_oid(¤t_oid)) {\n \t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n \t\t\t\t\t \"reference is missing but expected %s\"),\n \t\t\t\t ref_update_original_update_refname(u),\n \t\t\t\t oid_to_hex(&u->old_oid));\n-\t\telse\n+\t\t\treturn REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n+\t\t} else {\n \t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n \t\t\t\t\t \"is at %s but expected %s\"),\n \t\t\t\t ref_update_original_update_refname(u),\n \t\t\t\t oid_to_hex(¤t_oid),\n \t\t\t\t oid_to_hex(&u->old_oid));\n-\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t\t\treturn REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;\n+\t\t}\n \t}\n \n \t/*\n@@ -1296,8 +1297,8 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \tif ((u->type & REF_ISSYMREF) ||\n \t (u->flags & REF_LOG_ONLY) ||\n \t (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid)))\n-\t\treturn queue_transaction_update(refs, tx_data, u,\n-\t\t\t\t\t ¤t_oid, err);\n+\t\tif (queue_transaction_update(refs, tx_data, u, ¤t_oid, err))\n+\t\t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \n \treturn 0;\n }\n@@ -1385,7 +1386,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \ttransaction->state = REF_TRANSACTION_PREPARED;\n \n done:\n-\tassert(ret != REFTABLE_API_ERROR);\n \tif (ret < 0) {\n \t\tfree_transaction_data(tx_data);\n \t\ttransaction->state = REF_TRANSACTION_CLOSED;\n\n-- \n2.48.1\n\n"},{"id":"515180","messageId":"20250327-245-partially-atomic-ref-updates-v5-6-4db2a3e34404@gmail.com","threadId":"62915","inReplyTo":"20250327-245-partially-atomic-ref-updates-v5-0-4db2a3e34404@gmail.com","subject":"[PATCH v5 6/8] refs: implement batch reference update support","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-03-27T11:13:30Z","receivedAt":"2025-03-27T11:13:38Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Git supports making reference updates with or without transactions.\nUpdates with transactions are generally better optimized. But\ntransactions are all or nothing. This means, if a user wants to batch\nupdates to take advantage of the optimizations without the hard\nrequirement that all updates must succeed, there is no way currently to\ndo so. Particularly with the reftable backend where batching multiple\nreference updates is more efficient than performing them sequentially.\n\nIntroduce batched update support with a new flag,\n'REF_TRANSACTION_ALLOW_FAILURE'. Batched updates while different from\ntransactions, use the transaction infrastructure under the hood. When\nenabled, this flag allows individual reference updates that would\ntypically cause the entire transaction to fail due to non-system-related\nerrors to be marked as rejected while permitting other updates to\nproceed. System errors referred by 'REF_TRANSACTION_ERROR_GENERIC'\ncontinue to result in the entire transaction failing. This approach\nenhances flexibility while preserving transactional integrity where\nnecessary.\n\nThe implementation introduces several key components:\n\n - Add 'rejection_err' field to struct `ref_update` to track failed\n updates with failure reason.\n\n - Add a new struct `ref_transaction_rejections` and a field within\n `ref_transaction` to this struct to allow quick iteration over\n rejected updates.\n\n - Modify reference backends (files, packed, reftable) to handle\n partial transactions by using `ref_transaction_set_rejected()`\n instead of failing the entire transaction when\n `REF_TRANSACTION_ALLOW_FAILURE` is set.\n\n - Add `ref_transaction_for_each_rejected_update()` to let callers\n examine which updates were rejected and why.\n\nThis foundational change enables batched update support throughout the\nreference subsystem. A following commit will expose this capability to\nusers by adding a `--batch-updates` flag to 'git-update-ref(1)',\nproviding both a user-facing feature and a testable implementation.\n\nSigned-off-by: Karthik Nayak \n---\n refs.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++\n refs.h | 22 ++++++++++++++++++\n refs/files-backend.c | 12 +++++++++-\n refs/packed-backend.c | 27 ++++++++++++++++++++--\n refs/refs-internal.h | 26 +++++++++++++++++++++\n refs/reftable-backend.c | 12 +++++++++-\n 6 files changed, 156 insertions(+), 4 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 3d0b53d56e..b34ec198f5 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1176,6 +1176,10 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,\n \ttr->ref_store = refs;\n \ttr->flags = flags;\n \tstring_list_init_dup(&tr->refnames);\n+\n+\tif (flags & REF_TRANSACTION_ALLOW_FAILURE)\n+\t\tCALLOC_ARRAY(tr->rejections, 1);\n+\n \treturn tr;\n }\n \n@@ -1206,11 +1210,45 @@ void ref_transaction_free(struct ref_transaction *transaction)\n \t\tfree((char *)transaction->updates[i]->old_target);\n \t\tfree(transaction->updates[i]);\n \t}\n+\n+\tif (transaction->rejections)\n+\t\tfree(transaction->rejections->update_indices);\n+\tfree(transaction->rejections);\n+\n \tstring_list_clear(&transaction->refnames, 0);\n \tfree(transaction->updates);\n \tfree(transaction);\n }\n \n+int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction,\n+\t\t\t\t size_t update_idx,\n+\t\t\t\t enum ref_transaction_error err)\n+{\n+\tif (update_idx >= transaction->nr)\n+\t\tBUG(\"trying to set rejection on invalid update index\");\n+\n+\tif (!(transaction->flags & REF_TRANSACTION_ALLOW_FAILURE))\n+\t\treturn 0;\n+\n+\tif (!transaction->rejections)\n+\t\tBUG(\"transaction not inititalized with failure support\");\n+\n+\t/*\n+\t * Don't accept generic errors, since these errors are not user\n+\t * input related.\n+\t */\n+\tif (err == REF_TRANSACTION_ERROR_GENERIC)\n+\t\treturn 0;\n+\n+\ttransaction->updates[update_idx]->rejection_err = err;\n+\tALLOC_GROW(transaction->rejections->update_indices,\n+\t\t transaction->rejections->nr + 1,\n+\t\t transaction->rejections->alloc);\n+\ttransaction->rejections->update_indices[transaction->rejections->nr++] = update_idx;\n+\n+\treturn 1;\n+}\n+\n struct ref_update *ref_transaction_add_update(\n \t\tstruct ref_transaction *transaction,\n \t\tconst char *refname, unsigned int flags,\n@@ -1236,6 +1274,7 @@ struct ref_update *ref_transaction_add_update(\n \ttransaction->updates[transaction->nr++] = update;\n \n \tupdate->flags = flags;\n+\tupdate->rejection_err = 0;\n \n \tupdate->new_target = xstrdup_or_null(new_target);\n \tupdate->old_target = xstrdup_or_null(old_target);\n@@ -2728,6 +2767,28 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,\n \t}\n }\n \n+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,\n+\t\t\t\t\t ref_transaction_for_each_rejected_update_fn cb,\n+\t\t\t\t\t void *cb_data)\n+{\n+\tif (!transaction->rejections)\n+\t\treturn;\n+\n+\tfor (size_t i = 0; i < transaction->rejections->nr; i++) {\n+\t\tsize_t update_index = transaction->rejections->update_indices[i];\n+\t\tstruct ref_update *update = transaction->updates[update_index];\n+\n+\t\tif (!update->rejection_err)\n+\t\t\tcontinue;\n+\n+\t\tcb(update->refname,\n+\t\t (update->flags & REF_HAVE_OLD) ? &update->old_oid : NULL,\n+\t\t (update->flags & REF_HAVE_NEW) ? &update->new_oid : NULL,\n+\t\t update->old_target, update->new_target,\n+\t\t update->rejection_err, cb_data);\n+\t}\n+}\n+\n int refs_delete_refs(struct ref_store *refs, const char *logmsg,\n \t\t struct string_list *refnames, unsigned int flags)\n {\ndiff --git a/refs.h b/refs.h\nindex f009cdae7d..c48c800478 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -667,6 +667,13 @@ enum ref_transaction_flag {\n \t * either be absent or null_oid.\n \t */\n \tREF_TRANSACTION_FLAG_INITIAL = (1 << 0),\n+\n+\t/*\n+\t * The transaction mechanism by default fails all updates if any conflict\n+\t * is detected. This flag allows transactions to partially apply updates\n+\t * while rejecting updates which do not match the expected state.\n+\t */\n+\tREF_TRANSACTION_ALLOW_FAILURE = (1 << 1),\n };\n \n /*\n@@ -897,6 +904,21 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,\n \t\t\t\t\t ref_transaction_for_each_queued_update_fn cb,\n \t\t\t\t\t void *cb_data);\n \n+/*\n+ * Execute the given callback function for each of the reference updates which\n+ * have been rejected in the given transaction.\n+ */\n+typedef void ref_transaction_for_each_rejected_update_fn(const char *refname,\n+\t\t\t\t\t\t\t const struct object_id *old_oid,\n+\t\t\t\t\t\t\t const struct object_id *new_oid,\n+\t\t\t\t\t\t\t const char *old_target,\n+\t\t\t\t\t\t\t const char *new_target,\n+\t\t\t\t\t\t\t enum ref_transaction_error err,\n+\t\t\t\t\t\t\t void *cb_data);\n+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,\n+\t\t\t\t\t ref_transaction_for_each_rejected_update_fn cb,\n+\t\t\t\t\t void *cb_data);\n+\n /*\n * Free `*transaction` and all associated data.\n */\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 4f27f7652c..256c69b942 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -2852,8 +2852,15 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t\tret = lock_ref_for_update(refs, update, transaction,\n \t\t\t\t\t head_ref, &refnames_to_check,\n \t\t\t\t\t err);\n-\t\tif (ret)\n+\t\tif (ret) {\n+\t\t\tif (ref_transaction_maybe_set_rejected(transaction, i, ret)) {\n+\t\t\t\tstrbuf_reset(err);\n+\t\t\t\tret = 0;\n+\n+\t\t\t\tcontinue;\n+\t\t\t}\n \t\t\tgoto cleanup;\n+\t\t}\n \n \t\tif (update->flags & REF_DELETING &&\n \t\t !(update->flags & REF_LOG_ONLY) &&\n@@ -3151,6 +3158,9 @@ static int files_transaction_finish(struct ref_store *ref_store,\n \t\tstruct ref_update *update = transaction->updates[i];\n \t\tstruct ref_lock *lock = update->backend_data;\n \n+\t\tif (update->rejection_err)\n+\t\t\tcontinue;\n+\n \t\tif (update->flags & REF_NEEDS_COMMIT ||\n \t\t update->flags & REF_LOG_ONLY) {\n \t\t\tif (parse_and_write_reflog(refs, update, lock, err)) {\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex d90bd815a3..debca86a2b 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1327,10 +1327,11 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,\n * remain locked when it is done.\n */\n static enum ref_transaction_error write_with_updates(struct packed_ref_store *refs,\n-\t\t\t\t\t\t struct string_list *updates,\n+\t\t\t\t\t\t struct ref_transaction *transaction,\n \t\t\t\t\t\t struct strbuf *err)\n {\n \tenum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;\n+\tstruct string_list *updates = &transaction->refnames;\n \tstruct ref_iterator *iter = NULL;\n \tsize_t i;\n \tint ok;\n@@ -1411,6 +1412,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re\n \t\t\t\t\t\t \"reference already exists\",\n \t\t\t\t\t\t update->refname);\n \t\t\t\t\tret = REF_TRANSACTION_ERROR_CREATE_EXISTS;\n+\n+\t\t\t\t\tif (ref_transaction_maybe_set_rejected(transaction, i, ret)) {\n+\t\t\t\t\t\tstrbuf_reset(err);\n+\t\t\t\t\t\tret = 0;\n+\t\t\t\t\t\tcontinue;\n+\t\t\t\t\t}\n+\n \t\t\t\t\tgoto error;\n \t\t\t\t} else if (!oideq(&update->old_oid, iter->oid)) {\n \t\t\t\t\tstrbuf_addf(err, \"cannot update ref '%s': \"\n@@ -1419,6 +1427,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re\n \t\t\t\t\t\t oid_to_hex(iter->oid),\n \t\t\t\t\t\t oid_to_hex(&update->old_oid));\n \t\t\t\t\tret = REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;\n+\n+\t\t\t\t\tif (ref_transaction_maybe_set_rejected(transaction, i, ret)) {\n+\t\t\t\t\t\tstrbuf_reset(err);\n+\t\t\t\t\t\tret = 0;\n+\t\t\t\t\t\tcontinue;\n+\t\t\t\t\t}\n+\n \t\t\t\t\tgoto error;\n \t\t\t\t}\n \t\t\t}\n@@ -1456,6 +1471,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re\n \t\t\t\t\t update->refname,\n \t\t\t\t\t oid_to_hex(&update->old_oid));\n \t\t\t\tret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n+\n+\t\t\t\tif (ref_transaction_maybe_set_rejected(transaction, i, ret)) {\n+\t\t\t\t\tstrbuf_reset(err);\n+\t\t\t\t\tret = 0;\n+\t\t\t\t\tcontinue;\n+\t\t\t\t}\n+\n \t\t\t\tgoto error;\n \t\t\t}\n \t\t}\n@@ -1521,6 +1543,7 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re\n write_error:\n \tstrbuf_addf(err, \"error writing to %s: %s\",\n \t\t get_tempfile_path(refs->tempfile), strerror(errno));\n+\tret = REF_TRANSACTION_ERROR_GENERIC;\n \n error:\n \tref_iterator_free(iter);\n@@ -1679,7 +1702,7 @@ static int packed_transaction_prepare(struct ref_store *ref_store,\n \t\tdata->own_lock = 1;\n \t}\n \n-\tret = write_with_updates(refs, &transaction->refnames, err);\n+\tret = write_with_updates(refs, transaction, err);\n \tif (ret)\n \t\tgoto failure;\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 3f1d19abd9..73a5379b73 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -123,6 +123,12 @@ struct ref_update {\n \t */\n \tuint64_t index;\n \n+\t/*\n+\t * Used in batched reference updates to mark if a given update\n+\t * was rejected.\n+\t */\n+\tenum ref_transaction_error rejection_err;\n+\n \t/*\n \t * If this ref_update was split off of a symref update via\n \t * split_symref_update(), then this member points at that\n@@ -142,6 +148,13 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t struct object_id *oid, struct strbuf *referent,\n \t\t unsigned int *type, int *failure_errno);\n \n+/*\n+ * Mark a given update as rejected with a given reason.\n+ */\n+int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction,\n+\t\t\t\t size_t update_idx,\n+\t\t\t\t enum ref_transaction_error err);\n+\n /*\n * Add a ref_update with the specified properties to transaction, and\n * return a pointer to the new object. This function does not verify\n@@ -183,6 +196,18 @@ enum ref_transaction_state {\n \tREF_TRANSACTION_CLOSED = 2\n };\n \n+/*\n+ * Data structure to hold indices of updates which were rejected, for batched\n+ * reference updates. While the updates themselves hold the rejection error,\n+ * this structure allows a transaction to iterate only over the rejected\n+ * updates.\n+ */\n+struct ref_transaction_rejections {\n+\tsize_t *update_indices;\n+\tsize_t alloc;\n+\tsize_t nr;\n+};\n+\n /*\n * Data structure for holding a reference transaction, which can\n * consist of checks and updates to multiple references, carried out\n@@ -195,6 +220,7 @@ struct ref_transaction {\n \tsize_t alloc;\n \tsize_t nr;\n \tenum ref_transaction_state state;\n+\tstruct ref_transaction_rejections *rejections;\n \tvoid *backend_data;\n \tunsigned int flags;\n \tuint64_t max_index;\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex bd6b042103..5db4a108b9 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -1371,8 +1371,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\t\t transaction->updates[i],\n \t\t\t\t\t &refnames_to_check, head_type,\n \t\t\t\t\t &head_referent, &referent, err);\n-\t\tif (ret)\n+\t\tif (ret) {\n+\t\t\tif (ref_transaction_maybe_set_rejected(transaction, i, ret)) {\n+\t\t\t\tstrbuf_reset(err);\n+\t\t\t\tret = 0;\n+\n+\t\t\t\tcontinue;\n+\t\t\t}\n \t\t\tgoto done;\n+\t\t}\n \t}\n \n \tret = refs_verify_refnames_available(ref_store, &refnames_to_check,\n@@ -1454,6 +1461,9 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data\n \t\tstruct reftable_transaction_update *tx_update = &arg->updates[i];\n \t\tstruct ref_update *u = tx_update->update;\n \n+\t\tif (u->rejection_err)\n+\t\t\tcontinue;\n+\n \t\t/*\n \t\t * Write a reflog entry when updating a ref to point to\n \t\t * something new in either of the following cases:\n\n-- \n2.48.1\n\n"},{"id":"515182","messageId":"20250327-245-partially-atomic-ref-updates-v5-7-4db2a3e34404@gmail.com","threadId":"62915","inReplyTo":"20250327-245-partially-atomic-ref-updates-v5-0-4db2a3e34404@gmail.com","subject":"[PATCH v5 7/8] refs: support rejection in batch updates during F/D checks","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-03-27T11:13:31Z","receivedAt":"2025-03-27T11:13:39Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"The `refs_verify_refnames_available()` is used to batch check refnames\nfor F/D conflicts. While this is the more performant alternative than\nits individual version, it does not provide rejection capabilities on a\nsingle update level. For batched updates, this would mean a rejection of\nthe entire transaction whenever one reference has a F/D conflict.\n\nModify the function to call `ref_transaction_maybe_set_rejected()` to\ncheck if a single update can be rejected. Since this function is only\ninternally used within 'refs/' and we want to pass in a `struct\nref_transaction *` as a variable. We also move and mark\n`refs_verify_refnames_available()` to 'refs-internal.h' to be an\ninternal function.\n\nSigned-off-by: Karthik Nayak \n---\n refs.c | 37 ++++++++++++++++++++++++++++++++++---\n refs.h | 12 ------------\n refs/files-backend.c | 27 ++++++++++++++++++---------\n refs/refs-internal.h | 16 ++++++++++++++++\n refs/reftable-backend.c | 11 ++++++++---\n 5 files changed, 76 insertions(+), 27 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex b34ec198f5..41d6247e70 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2540,6 +2540,7 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs\n \t\t\t\t\t const struct string_list *refnames,\n \t\t\t\t\t const struct string_list *extras,\n \t\t\t\t\t const struct string_list *skip,\n+\t\t\t\t\t struct ref_transaction *transaction,\n \t\t\t\t\t unsigned int initial_transaction,\n \t\t\t\t\t struct strbuf *err)\n {\n@@ -2547,6 +2548,7 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct string_list_item *item;\n \tstruct ref_iterator *iter = NULL;\n+\tstruct strset conflicting_dirnames;\n \tstruct strset dirnames;\n \tint ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \n@@ -2557,9 +2559,11 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs\n \n \tassert(err);\n \n+\tstrset_init(&conflicting_dirnames);\n \tstrset_init(&dirnames);\n \n \tfor_each_string_list_item(item, refnames) {\n+\t\tconst size_t *update_idx = (size_t *)item->util;\n \t\tconst char *refname = item->string;\n \t\tconst char *extra_refname;\n \t\tstruct object_id oid;\n@@ -2597,14 +2601,30 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs\n \t\t\t\tcontinue;\n \n \t\t\tif (!initial_transaction &&\n-\t\t\t !refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n-\t\t\t\t\t &type, &ignore_errno)) {\n+\t\t\t (strset_contains(&conflicting_dirnames, dirname.buf) ||\n+\t\t\t !refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n+\t\t\t\t\t\t &type, &ignore_errno))) {\n+\t\t\t\tif (transaction && ref_transaction_maybe_set_rejected(\n+\t\t\t\t\t transaction, *update_idx,\n+\t\t\t\t\t REF_TRANSACTION_ERROR_NAME_CONFLICT)) {\n+\t\t\t\t\tstrset_remove(&dirnames, dirname.buf);\n+\t\t\t\t\tstrset_add(&conflicting_dirnames, dirname.buf);\n+\t\t\t\t\tcontinue;\n+\t\t\t\t}\n+\n \t\t\t\tstrbuf_addf(err, _(\"'%s' exists; cannot create '%s'\"),\n \t\t\t\t\t dirname.buf, refname);\n \t\t\t\tgoto cleanup;\n \t\t\t}\n \n \t\t\tif (extras && string_list_has_string(extras, dirname.buf)) {\n+\t\t\t\tif (transaction && ref_transaction_maybe_set_rejected(\n+\t\t\t\t\t transaction, *update_idx,\n+\t\t\t\t\t REF_TRANSACTION_ERROR_NAME_CONFLICT)) {\n+\t\t\t\t\tstrset_remove(&dirnames, dirname.buf);\n+\t\t\t\t\tcontinue;\n+\t\t\t\t}\n+\n \t\t\t\tstrbuf_addf(err, _(\"cannot process '%s' and '%s' at the same time\"),\n \t\t\t\t\t refname, dirname.buf);\n \t\t\t\tgoto cleanup;\n@@ -2637,6 +2657,11 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs\n \t\t\t\t string_list_has_string(skip, iter->refname))\n \t\t\t\t\tcontinue;\n \n+\t\t\t\tif (transaction && ref_transaction_maybe_set_rejected(\n+\t\t\t\t\t transaction, *update_idx,\n+\t\t\t\t\t REF_TRANSACTION_ERROR_NAME_CONFLICT))\n+\t\t\t\t\tcontinue;\n+\n \t\t\t\tstrbuf_addf(err, _(\"'%s' exists; cannot create '%s'\"),\n \t\t\t\t\t iter->refname, refname);\n \t\t\t\tgoto cleanup;\n@@ -2648,6 +2673,11 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs\n \n \t\textra_refname = find_descendant_ref(dirname.buf, extras, skip);\n \t\tif (extra_refname) {\n+\t\t\tif (transaction && ref_transaction_maybe_set_rejected(\n+\t\t\t\t transaction, *update_idx,\n+\t\t\t\t REF_TRANSACTION_ERROR_NAME_CONFLICT))\n+\t\t\t\tcontinue;\n+\n \t\t\tstrbuf_addf(err, _(\"cannot process '%s' and '%s' at the same time\"),\n \t\t\t\t refname, extra_refname);\n \t\t\tgoto cleanup;\n@@ -2659,6 +2689,7 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs\n cleanup:\n \tstrbuf_release(&referent);\n \tstrbuf_release(&dirname);\n+\tstrset_clear(&conflicting_dirnames);\n \tstrset_clear(&dirnames);\n \tref_iterator_free(iter);\n \treturn ret;\n@@ -2679,7 +2710,7 @@ enum ref_transaction_error refs_verify_refname_available(\n \t};\n \n \treturn refs_verify_refnames_available(refs, &refnames, extras, skip,\n-\t\t\t\t\t initial_transaction, err);\n+\t\t\t\t\t NULL, initial_transaction, err);\n }\n \n struct do_for_each_reflog_help {\ndiff --git a/refs.h b/refs.h\nindex c48c800478..46a6008e07 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -141,18 +141,6 @@ enum ref_transaction_error refs_verify_refname_available(struct ref_store *refs,\n \t\t\t\t\t\t unsigned int initial_transaction,\n \t\t\t\t\t\t struct strbuf *err);\n \n-/*\n- * Same as `refs_verify_refname_available()`, but checking for a list of\n- * refnames instead of only a single item. This is more efficient in the case\n- * where one needs to check multiple refnames.\n- */\n-enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,\n-\t\t\t\t\t const struct string_list *refnames,\n-\t\t\t\t\t const struct string_list *extras,\n-\t\t\t\t\t const struct string_list *skip,\n-\t\t\t\t\t unsigned int initial_transaction,\n-\t\t\t\t\t struct strbuf *err);\n-\n int refs_ref_exists(struct ref_store *refs, const char *refname);\n \n int should_autocreate_reflog(enum log_refs_config log_all_ref_updates,\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 256c69b942..b96a511977 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -677,16 +677,18 @@ static void unlock_ref(struct ref_lock *lock)\n * - Generate informative error messages in the case of failure\n */\n static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,\n-\t\t\t\t\t const char *refname,\n+\t\t\t\t\t struct ref_update *update,\n+\t\t\t\t\t size_t update_idx,\n \t\t\t\t\t int mustexist,\n \t\t\t\t\t struct string_list *refnames_to_check,\n \t\t\t\t\t const struct string_list *extras,\n \t\t\t\t\t struct ref_lock **lock_p,\n \t\t\t\t\t struct strbuf *referent,\n-\t\t\t\t\t unsigned int *type,\n \t\t\t\t\t struct strbuf *err)\n {\n \tenum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;\n+\tconst char *refname = update->refname;\n+\tunsigned int *type = &update->type;\n \tstruct ref_lock *lock;\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tint attempts_remaining = 3;\n@@ -785,6 +787,8 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,\n \n \tif (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,\n \t\t\t type, &failure_errno)) {\n+\t\tstruct string_list_item *item;\n+\n \t\tif (failure_errno == ENOENT) {\n \t\t\tif (mustexist) {\n \t\t\t\t/* Garden variety missing reference. */\n@@ -864,7 +868,9 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,\n \t\t * make sure there is no existing packed ref that conflicts\n \t\t * with refname. This check is deferred so that we can batch it.\n \t\t */\n-\t\tstring_list_append(refnames_to_check, refname);\n+\t\titem = string_list_append(refnames_to_check, refname);\n+\t\titem->util = xmalloc(sizeof(update_idx));\n+\t\tmemcpy(item->util, &update_idx, sizeof(update_idx));\n \t}\n \n \tret = 0;\n@@ -2547,6 +2553,7 @@ struct files_transaction_backend_data {\n */\n static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *refs,\n \t\t\t\t\t\t struct ref_update *update,\n+\t\t\t\t\t\t size_t update_idx,\n \t\t\t\t\t\t struct ref_transaction *transaction,\n \t\t\t\t\t\t const char *head_ref,\n \t\t\t\t\t\t struct string_list *refnames_to_check,\n@@ -2575,9 +2582,9 @@ static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *re\n \tif (lock) {\n \t\tlock->count++;\n \t} else {\n-\t\tret = lock_raw_ref(refs, update->refname, mustexist,\n+\t\tret = lock_raw_ref(refs, update, update_idx, mustexist,\n \t\t\t\t refnames_to_check, &transaction->refnames,\n-\t\t\t\t &lock, &referent, &update->type, err);\n+\t\t\t\t &lock, &referent, err);\n \t\tif (ret) {\n \t\t\tchar *reason;\n \n@@ -2849,7 +2856,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n \n-\t\tret = lock_ref_for_update(refs, update, transaction,\n+\t\tret = lock_ref_for_update(refs, update, i, transaction,\n \t\t\t\t\t head_ref, &refnames_to_check,\n \t\t\t\t\t err);\n \t\tif (ret) {\n@@ -2905,7 +2912,8 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t * So instead, we accept the race for now.\n \t */\n \tif (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,\n-\t\t\t\t\t &transaction->refnames, NULL, 0, err)) {\n+\t\t\t\t\t &transaction->refnames, NULL, transaction,\n+\t\t\t\t\t 0, err)) {\n \t\tret = REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t\tgoto cleanup;\n \t}\n@@ -2951,7 +2959,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \n cleanup:\n \tfree(head_ref);\n-\tstring_list_clear(&refnames_to_check, 0);\n+\tstring_list_clear(&refnames_to_check, 1);\n \n \tif (ret)\n \t\tfiles_transaction_cleanup(refs, transaction);\n@@ -3097,7 +3105,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \t}\n \n \tif (refs_verify_refnames_available(&refs->base, &refnames_to_check,\n-\t\t\t\t\t &affected_refnames, NULL, 1, err)) {\n+\t\t\t\t\t &affected_refnames, NULL, transaction,\n+\t\t\t\t\t 1, err)) {\n \t\tpacked_refs_unlock(refs->packed_ref_store);\n \t\tret = REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t\tgoto cleanup;\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 73a5379b73..f868870851 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -806,4 +806,20 @@ enum ref_transaction_error ref_update_check_old_target(const char *referent,\n */\n int ref_update_expects_existing_old_ref(struct ref_update *update);\n \n+/*\n+ * Same as `refs_verify_refname_available()`, but checking for a list of\n+ * refnames instead of only a single item. This is more efficient in the case\n+ * where one needs to check multiple refnames.\n+ *\n+ * If using batched updates, then individual updates are marked rejected,\n+ * reference backends are then in charge of not committing those updates.\n+ */\n+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,\n+\t\t\t\t\t const struct string_list *refnames,\n+\t\t\t\t\t const struct string_list *extras,\n+\t\t\t\t\t const struct string_list *skip,\n+\t\t\t\t\t struct ref_transaction *transaction,\n+\t\t\t\t\t unsigned int initial_transaction,\n+\t\t\t\t\t struct strbuf *err);\n+\n #endif /* REFS_REFS_INTERNAL_H */\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex 5db4a108b9..4c3817f4ec 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -1074,6 +1074,7 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor\n \t\t\t\t\t\t\tstruct ref_transaction *transaction,\n \t\t\t\t\t\t\tstruct reftable_backend *be,\n \t\t\t\t\t\t\tstruct ref_update *u,\n+\t\t\t\t\t\t\tsize_t update_idx,\n \t\t\t\t\t\t\tstruct string_list *refnames_to_check,\n \t\t\t\t\t\t\tunsigned int head_type,\n \t\t\t\t\t\t\tstruct strbuf *head_referent,\n@@ -1149,6 +1150,7 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor\n \tif (ret < 0)\n \t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \tif (ret > 0 && !ref_update_expects_existing_old_ref(u)) {\n+\t\tstruct string_list_item *item;\n \t\t/*\n \t\t * The reference does not exist, and we either have no\n \t\t * old object ID or expect the reference to not exist.\n@@ -1158,7 +1160,9 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor\n \t\t * can output a proper error message instead of failing\n \t\t * at a later point.\n \t\t */\n-\t\tstring_list_append(refnames_to_check, u->refname);\n+\t\titem = string_list_append(refnames_to_check, u->refname);\n+\t\titem->util = xmalloc(sizeof(update_idx));\n+\t\tmemcpy(item->util, &update_idx, sizeof(update_idx));\n \n \t\t/*\n \t\t * There is no need to write the reference deletion\n@@ -1368,7 +1372,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tret = prepare_single_update(refs, tx_data, transaction, be,\n-\t\t\t\t\t transaction->updates[i],\n+\t\t\t\t\t transaction->updates[i], i,\n \t\t\t\t\t &refnames_to_check, head_type,\n \t\t\t\t\t &head_referent, &referent, err);\n \t\tif (ret) {\n@@ -1384,6 +1388,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \n \tret = refs_verify_refnames_available(ref_store, &refnames_to_check,\n \t\t\t\t\t &transaction->refnames, NULL,\n+\t\t\t\t\t transaction,\n \t\t\t\t\t transaction->flags & REF_TRANSACTION_FLAG_INITIAL,\n \t\t\t\t\t err);\n \tif (ret < 0)\n@@ -1402,7 +1407,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t}\n \tstrbuf_release(&referent);\n \tstrbuf_release(&head_referent);\n-\tstring_list_clear(&refnames_to_check, 0);\n+\tstring_list_clear(&refnames_to_check, 1);\n \n \treturn ret;\n }\n\n-- \n2.48.1\n\n"},{"id":"515183","messageId":"20250327-245-partially-atomic-ref-updates-v5-8-4db2a3e34404@gmail.com","threadId":"62915","inReplyTo":"20250327-245-partially-atomic-ref-updates-v5-0-4db2a3e34404@gmail.com","subject":"[PATCH v5 8/8] update-ref: add --batch-updates flag for stdin mode","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-03-27T11:13:32Z","receivedAt":"2025-03-27T11:13:40Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"When updating multiple references through stdin, Git's update-ref\ncommand normally aborts the entire transaction if any single update\nfails. This atomic behavior prevents partial updates. Introduce a new\nbatch update system, where the updates the performed together similar\nbut individual updates are allowed to fail.\n\nAdd a new `--batch-updates` flag that allows the transaction to continue\neven when individual reference updates fail. This flag can only be used\nin `--stdin` mode and builds upon the batch update support added to the\nrefs subsystem in the previous commits. When enabled, failed updates are\nreported in the following format:\n\n rejected SP ( | ) SP ( | ) SP LF\n\nUpdate the documentation to reflect this change and also tests to cover\ndifferent scenarios where an update could be rejected.\n\nSigned-off-by: Karthik Nayak \n---\n Documentation/git-update-ref.adoc | 14 ++-\n builtin/update-ref.c | 66 ++++++++++-\n t/t1400-update-ref.sh | 233 ++++++++++++++++++++++++++++++++++++++\n 3 files changed, 306 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/git-update-ref.adoc b/Documentation/git-update-ref.adoc\nindex 9e6935d38d..5be2c16776 100644\n--- a/Documentation/git-update-ref.adoc\n+++ b/Documentation/git-update-ref.adoc\n@@ -7,8 +7,10 @@ git-update-ref - Update the object name stored in a ref safely\n \n SYNOPSIS\n --------\n-[verse]\n-'git update-ref' [-m ] [--no-deref] (-d [] | [--create-reflog] [] | --stdin [-z])\n+[synopsis]\n+git update-ref [-m ] [--no-deref] -d []\n+\t [-m ] [--no-deref] [--create-reflog] []\n+ [-m ] [--no-deref] --stdin [-z] [--batch-updates]\n \n DESCRIPTION\n -----------\n@@ -57,6 +59,14 @@ performs all modifications together. Specify commands of the form:\n With `--create-reflog`, update-ref will create a reflog for each ref\n even if one would not ordinarily be created.\n \n+With `--batch-updates`, update-ref executes the updates in a batch but allows\n+individual updates to fail due to invalid or incorrect user input, applying only\n+the successful updates. However, system-related errors—such as I/O failures or\n+memory issues—will result in a full failure of all batched updates. Any failed\n+updates will be reported in the following format:\n+\n+\trejected SP ( | ) SP ( | ) SP LF\n+\n Quote fields containing whitespace as if they were strings in C source\n code; i.e., surrounded by double-quotes and with backslash escapes.\n Use 40 \"0\" characters or the empty string to specify a zero value. To\ndiff --git a/builtin/update-ref.c b/builtin/update-ref.c\nindex 1d541e13ad..111d6473ad 100644\n--- a/builtin/update-ref.c\n+++ b/builtin/update-ref.c\n@@ -5,6 +5,7 @@\n #include \"config.h\"\n #include \"gettext.h\"\n #include \"hash.h\"\n+#include \"hex.h\"\n #include \"refs.h\"\n #include \"object-name.h\"\n #include \"parse-options.h\"\n@@ -13,7 +14,7 @@\n static const char * const git_update_ref_usage[] = {\n \tN_(\"git update-ref [] -d []\"),\n \tN_(\"git update-ref [] []\"),\n-\tN_(\"git update-ref [] --stdin [-z]\"),\n+\tN_(\"git update-ref [] --stdin [-z] [--batch-updates]\"),\n \tNULL\n };\n \n@@ -565,6 +566,49 @@ static void parse_cmd_abort(struct ref_transaction *transaction,\n \treport_ok(\"abort\");\n }\n \n+static void print_rejected_refs(const char *refname,\n+\t\t\t\tconst struct object_id *old_oid,\n+\t\t\t\tconst struct object_id *new_oid,\n+\t\t\t\tconst char *old_target,\n+\t\t\t\tconst char *new_target,\n+\t\t\t\tenum ref_transaction_error err,\n+\t\t\t\tvoid *cb_data UNUSED)\n+{\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tconst char *reason = \"\";\n+\n+\tswitch (err) {\n+\tcase REF_TRANSACTION_ERROR_NAME_CONFLICT:\n+\t\treason = \"refname conflict\";\n+\t\tbreak;\n+\tcase REF_TRANSACTION_ERROR_CREATE_EXISTS:\n+\t\treason = \"reference already exists\";\n+\t\tbreak;\n+\tcase REF_TRANSACTION_ERROR_NONEXISTENT_REF:\n+\t\treason = \"reference does not exist\";\n+\t\tbreak;\n+\tcase REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE:\n+\t\treason = \"incorrect old value provided\";\n+\t\tbreak;\n+\tcase REF_TRANSACTION_ERROR_INVALID_NEW_VALUE:\n+\t\treason = \"invalid new value provided\";\n+\t\tbreak;\n+\tcase REF_TRANSACTION_ERROR_EXPECTED_SYMREF:\n+\t\treason = \"expected symref but found regular ref\";\n+\t\tbreak;\n+\tdefault:\n+\t\treason = \"unkown failure\";\n+\t}\n+\n+\tstrbuf_addf(&sb, \"rejected %s %s %s %s\\n\", refname,\n+\t\t new_oid ? oid_to_hex(new_oid) : new_target,\n+\t\t old_oid ? oid_to_hex(old_oid) : old_target,\n+\t\t reason);\n+\n+\tfwrite(sb.buf, sb.len, 1, stdout);\n+\tstrbuf_release(&sb);\n+}\n+\n static void parse_cmd_commit(struct ref_transaction *transaction,\n \t\t\t const char *next, const char *end UNUSED)\n {\n@@ -573,6 +617,10 @@ static void parse_cmd_commit(struct ref_transaction *transaction,\n \t\tdie(\"commit: extra input: %s\", next);\n \tif (ref_transaction_commit(transaction, &error))\n \t\tdie(\"commit: %s\", error.buf);\n+\n+\tref_transaction_for_each_rejected_update(transaction,\n+\t\t\t\t\t\t print_rejected_refs, NULL);\n+\n \treport_ok(\"commit\");\n \tref_transaction_free(transaction);\n }\n@@ -609,7 +657,7 @@ static const struct parse_cmd {\n \t{ \"commit\", parse_cmd_commit, 0, UPDATE_REFS_CLOSED },\n };\n \n-static void update_refs_stdin(void)\n+static void update_refs_stdin(unsigned int flags)\n {\n \tstruct strbuf input = STRBUF_INIT, err = STRBUF_INIT;\n \tenum update_refs_state state = UPDATE_REFS_OPEN;\n@@ -617,7 +665,7 @@ static void update_refs_stdin(void)\n \tint i, j;\n \n \ttransaction = ref_store_transaction_begin(get_main_ref_store(the_repository),\n-\t\t\t\t\t\t 0, &err);\n+\t\t\t\t\t\t flags, &err);\n \tif (!transaction)\n \t\tdie(\"%s\", err.buf);\n \n@@ -685,7 +733,7 @@ static void update_refs_stdin(void)\n \t\t\t */\n \t\t\tstate = cmd->state;\n \t\t\ttransaction = ref_store_transaction_begin(get_main_ref_store(the_repository),\n-\t\t\t\t\t\t\t\t 0, &err);\n+\t\t\t\t\t\t\t\t flags, &err);\n \t\t\tif (!transaction)\n \t\t\t\tdie(\"%s\", err.buf);\n \n@@ -701,6 +749,8 @@ static void update_refs_stdin(void)\n \t\t/* Commit by default if no transaction was requested. */\n \t\tif (ref_transaction_commit(transaction, &err))\n \t\t\tdie(\"%s\", err.buf);\n+\t\tref_transaction_for_each_rejected_update(transaction,\n+\t\t\t\t\t\t print_rejected_refs, NULL);\n \t\tref_transaction_free(transaction);\n \t\tbreak;\n \tcase UPDATE_REFS_STARTED:\n@@ -727,6 +777,8 @@ int cmd_update_ref(int argc,\n \tstruct object_id oid, oldoid;\n \tint delete = 0, no_deref = 0, read_stdin = 0, end_null = 0;\n \tint create_reflog = 0;\n+\tunsigned int flags = 0;\n+\n \tstruct option options[] = {\n \t\tOPT_STRING( 'm', NULL, &msg, N_(\"reason\"), N_(\"reason of the update\")),\n \t\tOPT_BOOL('d', NULL, &delete, N_(\"delete the reference\")),\n@@ -735,6 +787,8 @@ int cmd_update_ref(int argc,\n \t\tOPT_BOOL('z', NULL, &end_null, N_(\"stdin has NUL-terminated arguments\")),\n \t\tOPT_BOOL( 0 , \"stdin\", &read_stdin, N_(\"read updates from stdin\")),\n \t\tOPT_BOOL( 0 , \"create-reflog\", &create_reflog, N_(\"create a reflog\")),\n+\t\tOPT_BIT('0', \"batch-updates\", &flags, N_(\"batch reference updates\"),\n+\t\t\tREF_TRANSACTION_ALLOW_FAILURE),\n \t\tOPT_END(),\n \t};\n \n@@ -756,8 +810,10 @@ int cmd_update_ref(int argc,\n \t\t\tusage_with_options(git_update_ref_usage, options);\n \t\tif (end_null)\n \t\t\tline_termination = '\\0';\n-\t\tupdate_refs_stdin();\n+\t\tupdate_refs_stdin(flags);\n \t\treturn 0;\n+\t} else if (flags & REF_TRANSACTION_ALLOW_FAILURE) {\n+\t\tdie(\"--batch-updates can only be used with --stdin\");\n \t}\n \n \tif (end_null)\ndiff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh\nindex 29045aad43..d29d23cb89 100755\n--- a/t/t1400-update-ref.sh\n+++ b/t/t1400-update-ref.sh\n@@ -2066,6 +2066,239 @@ do\n \t\tgrep \"$(git rev-parse $a) $(git rev-parse $a)\" actual\n \t'\n \n+\ttest_expect_success \"stdin $type batch-updates\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit commit &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$head\" \"$Z\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$head\" \"$Z\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --batch-updates expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\tgit rev-parse refs/heads/ref2 >actual &&\n+\t\t\ttest_cmp expect actual\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type batch-updates with invalid new_oid\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\t\t\tgit update-ref refs/heads/ref2 $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$(test_oid 001)\" \"$head\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --batch-updates stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\techo $head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref2 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"invalid new value provided\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type batch-updates with non-commit new_oid\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\thead_tree=$(git rev-parse HEAD^{tree}) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\t\t\tgit update-ref refs/heads/ref2 $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$head_tree\" \"$head\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --batch-updates stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\techo $head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref2 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"invalid new value provided\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type batch-updates with non-existent ref\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$old_head\" \"$head\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --batch-updates stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_must_fail git rev-parse refs/heads/ref2 &&\n+\t\t\ttest_grep -q \"reference does not exist\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type batch-updates with dangling symref\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\t\t\tgit symbolic-ref refs/heads/ref2 refs/heads/nonexistent &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$old_head\" \"$head\" >>stdin &&\n+\t\t\tgit update-ref $type --no-deref --stdin --batch-updates stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\techo $head >expect &&\n+\t\t\ttest_must_fail git rev-parse refs/heads/ref2 &&\n+\t\t\ttest_grep -q \"reference does not exist\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type batch-updates with regular ref as symref\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\t\t\tgit update-ref refs/heads/ref2 $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"symref-update refs/heads/ref2\" \"$old_head\" \"ref\" \"refs/heads/nonexistent\" >>stdin &&\n+\t\t\tgit update-ref $type --no-deref --stdin --batch-updates stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\techo $head >expect &&\n+\t\t\techo $head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref2 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"expected symref but found regular ref\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type batch-updates with invalid old_oid\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\t\t\tgit update-ref refs/heads/ref2 $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$old_head\" \"$Z\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --batch-updates stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\techo $head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref2 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"reference already exists\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type batch-updates with incorrect old oid\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\t\t\tgit update-ref refs/heads/ref2 $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$head\" \"$old_head\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --batch-updates stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\techo $head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref2 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"incorrect old value provided\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type batch-updates refname conflict\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref/foo $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref/foo\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref\" \"$old_head\" \"\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --batch-updates stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref/foo >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"refname conflict\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type batch-updates refname conflict new ref\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref/foo $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/foo\" \"$old_head\" \"\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref\" \"$old_head\" \"\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --batch-updates stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/foo >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"refname conflict\" stdout\n+\t\t)\n+\t'\n done\n \n test_expect_success 'update-ref should also create reflog for HEAD' '\n\n-- \n2.48.1\n\n"},{"id":"515223","messageId":"Z-ZqvbK9gJtuo_mS@pks.im","threadId":"62915","inReplyTo":"20250327-245-partially-atomic-ref-updates-v5-0-4db2a3e34404@gmail.com","subject":"Re: [PATCH v5 0/8] refs: introduce support for batched reference updates","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-28T09:24:13Z","receivedAt":"2025-03-28T09:24:17Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Mar 27, 2025 at 12:13:24PM +0100, Karthik Nayak wrote:\n> Changes in v5:\n> - Inline the comments around the 'ref_transaction_error'.\n> - Use 'strbuf_reset()' wherever possible instead of 'strbuf_setlen(err, 0)'.\n> - Use an extra 'conflicting_dirnames' strset in 'refs_verify_refnames_available()' to track\n> dirnames which were found to be conflicting, this is to avoid re-reading those dirnames.\n> - Add curly braces style mismatch in if..else block.\n> - Link to v4: https://lore.kernel.org/r/20250320-245-partially-atomic-ref-updates-v4-0-3dcc1b311dc9@gmail.com\n\nThanks, the series looks good to me judging by the range diff.\n\nPatrick\n"},{"id":"515231","messageId":"6141866.lOV4Wx5bFT@cayenne","threadId":"62915","inReplyTo":"20250327-245-partially-atomic-ref-updates-v5-8-4db2a3e34404@gmail.com","subject":"Re: [PATCH v5 8/8] update-ref: add --batch-updates flag for stdin mode","fromName":"Jean-Noël AVILA","fromEmail":"jn.avila@free.fr","sentAt":"2025-03-28T13:00:49Z","receivedAt":"2025-03-28T13:09:48Z","isPatch":true,"sender":{"key":"jn.avila@free.fr","avatar":"https://avatars.githubusercontent.com/u/156172?v=4"},"body":"On Thursday, 27 March 2025 12:13:32 CET Karthik Nayak wrote:\n> When updating multiple references through stdin, Git's update-ref\n> command normally aborts the entire transaction if any single update\n> fails. This atomic behavior prevents partial updates. Introduce a new\n> batch update system, where the updates the performed together similar\n> but individual updates are allowed to fail.\n> \n> Add a new `--batch-updates` flag that allows the transaction to continue\n> even when individual reference updates fail. This flag can only be used\n> in `--stdin` mode and builds upon the batch update support added to the\n> refs subsystem in the previous commits. When enabled, failed updates are\n> reported in the following format:\n> \n> rejected SP ( | ) SP ( | ) SP\n> LF\n> \n> Update the documentation to reflect this change and also tests to cover\n> different scenarios where an update could be rejected.\n> \n> Signed-off-by: Karthik Nayak \n> ---\n> Documentation/git-update-ref.adoc | 14 ++-\n> builtin/update-ref.c | 66 ++++++++++-\n> t/t1400-update-ref.sh | 233\n> ++++++++++++++++++++++++++++++++++++++ 3 files changed, 306 insertions(+), 7\n> deletions(-)\n> \n> diff --git a/Documentation/git-update-ref.adoc\n> b/Documentation/git-update-ref.adoc index 9e6935d38d..5be2c16776 100644\n> --- a/Documentation/git-update-ref.adoc\n> +++ b/Documentation/git-update-ref.adoc\n> @@ -7,8 +7,10 @@ git-update-ref - Update the object name stored in a ref\n> safely\n> \n> SYNOPSIS\n> --------\n> -[verse]\n> -'git update-ref' [-m ] [--no-deref] (-d [] |\n> [--create-reflog] [] | --stdin [-z]) +[synopsis]\n> +git update-ref [-m ] [--no-deref] -d []\n> +\t [-m ] [--no-deref] [--create-reflog] \n> [] + [-m ] [--no-deref] --stdin [-z]\n> [--batch-updates]\n\nIn the case of expressing alternative command line invocations, you need to \nrepeat the \"git update-ref\" command on each line. Otherwise, it means that \nthis is the continuation of possible options of one command\n\n\n\n> \n> DESCRIPTION\n> -----------\n\n\n\n"},{"id":"515277","messageId":"xmqq34evequa.fsf@gitster.g","threadId":"62915","inReplyTo":"6141866.lOV4Wx5bFT@cayenne","subject":"Re: [PATCH v5 8/8] update-ref: add --batch-updates flag for stdin mode","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-03-29T16:36:45Z","receivedAt":"2025-03-29T16:36:47Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jean-Noël AVILA writes:\n\n>> SYNOPSIS\n>> --------\n>> -[verse]\n>> -'git update-ref' [-m ] [--no-deref] (-d [] |\n>> [--create-reflog] [] | --stdin [-z]) +[synopsis]\n>> +git update-ref [-m ] [--no-deref] -d []\n>> +\t [-m ] [--no-deref] [--create-reflog] \n>> [] + [-m ] [--no-deref] --stdin [-z]\n>> [--batch-updates]\n>\n> In the case of expressing alternative command line invocations, you need to \n> repeat the \"git update-ref\" command on each line. Otherwise, it means that \n> this is the continuation of possible options of one command\n\n\nLike this?\n\ndiff --git a/Documentation/git-update-ref.adoc b/Documentation/git-update-ref.adoc\nindex 5be2c16776..9310ce9768 100644\n--- a/Documentation/git-update-ref.adoc\n+++ b/Documentation/git-update-ref.adoc\n@@ -9,8 +9,8 @@ SYNOPSIS\n --------\n [synopsis]\n git update-ref [-m ] [--no-deref] -d []\n-\t [-m ] [--no-deref] [--create-reflog] []\n- [-m ] [--no-deref] --stdin [-z] [--batch-updates]\n+git update-ref [-m ] [--no-deref] [--create-reflog] []\n+git update-ref [-m ] [--no-deref] --stdin [-z] [--batch-updates]\n \n DESCRIPTION\n -----------\n"},{"id":"515280","messageId":"CAOLa=ZQDiCx_VJRbQB1K6EeuA2mDj4twEB98ByddUNLtB8rEKQ@mail.gmail.com","threadId":"62915","inReplyTo":"xmqq34evequa.fsf@gitster.g","subject":"Re: [PATCH v5 8/8] update-ref: add --batch-updates flag for stdin mode","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-03-29T18:18:52Z","receivedAt":"2025-03-29T18:18:53Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Junio C Hamano writes:\n\n> Jean-Noël AVILA writes:\n>\n>>> SYNOPSIS\n>>> --------\n>>> -[verse]\n>>> -'git update-ref' [-m ] [--no-deref] (-d [] |\n>>> [--create-reflog] [] | --stdin [-z]) +[synopsis]\n>>> +git update-ref [-m ] [--no-deref] -d []\n>>> +\t [-m ] [--no-deref] [--create-reflog] \n>>> [] + [-m ] [--no-deref] --stdin [-z]\n>>> [--batch-updates]\n>>\n>> In the case of expressing alternative command line invocations, you need to\n>> repeat the \"git update-ref\" command on each line. Otherwise, it means that\n>> this is the continuation of possible options of one command\n>\n>\n> Like this?\n>\n> diff --git a/Documentation/git-update-ref.adoc b/Documentation/git-update-ref.adoc\n> index 5be2c16776..9310ce9768 100644\n> --- a/Documentation/git-update-ref.adoc\n> +++ b/Documentation/git-update-ref.adoc\n> @@ -9,8 +9,8 @@ SYNOPSIS\n> --------\n> [synopsis]\n> git update-ref [-m ] [--no-deref] -d []\n> -\t [-m ] [--no-deref] [--create-reflog] []\n> - [-m ] [--no-deref] --stdin [-z] [--batch-updates]\n> +git update-ref [-m ] [--no-deref] [--create-reflog] []\n> +git update-ref [-m ] [--no-deref] --stdin [-z] [--batch-updates]\n>\n> DESCRIPTION\n> -----------\n\nI think you also caught a whitespace issue here! I'll add this locally\nnevertheless, but will hold on re-rolling! Thanks both\n"},{"id":"515835","messageId":"20250408085120.614893-1-karthik.188@gmail.com","threadId":"62915","inReplyTo":"20250207-245-partially-atomic-ref-updates-v1-0-e6a3690ff23a@gmail.com","subject":"[PATCH v6 0/8] refs: introduce support for batched reference updates","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-08T08:51:04Z","receivedAt":"2025-04-08T08:51:25Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Git supports making reference updates with or without transactions.\nUpdates with transactions are generally better optimized. But\ntransactions are all or nothing. This means, if a user wants to batch\nupdates to take advantage of the optimizations without the hard\nrequirement that all updates must succeed, there is no way currently to\ndo so. Particularly with the reftable backend where batching multiple\nreference updates is more efficient than performing them sequentially.\n\nThis series introduces support for batched reference updates without\ntransactions allowing individual reference updates to fail while letting\nothers proceed. This capability is exposed through git-update-ref's\n`--allow-partial` flag, which can be used in `--stdin` mode to batch\nupdates and handle failures gracefully. Under the hood, these batched\nupdates still use the transactions infrastructure, while modifying\nsections to allow partial failures.\n\nThe changes are structured to carefully build up this functionality:\n\nFirst, we clean up and consolidate the reference update checking logic.\nThis includes removing duplicate checks in the files backend and moving\nrefname tracking to the generic layer, which simplifies the codebase and\nprepares it for the new feature.\n\nWe then restructure the reftable backend's transaction preparation code,\nextracting the update validation logic into a dedicated function. This\nnot only improves code organization but sets the stage for implementing\npartial transaction support.\n\nTo ensure we only skip errors which are user-oriented, we introduce\ntyped errors for transactions with 'enum ref_transaction_error'. We\nextend the existing errors to include other scenarios and use this new\nerrors throughout the refs code.\n\nWith this groundwork in place, we implement the core batch update\nsupport in the refs subsystem. This adds the necessary infrastructure to\ntrack and report rejected updates while allowing transactions to\nproceed. All reference backends are modified to support this behavior\nwhen enabled.\n\nFinally, we expose this functionality to users through\ngit-update-ref(1)'s `--allow-partial` flag, complete with test coverage\nand documentation. The flag is specifically limited to `--stdin` mode\nwhere batching multiple updates is most relevant.\n\nThis enhancement improves Git's flexibility in handling reference\nupdates while maintaining the safety of atomic transactions by default.\nIt's particularly valuable for tools and workflows that need to handle\nreference update failures gracefully without abandoning the entire batch\nof updates.\n\nThis series is based on top of 683c54c999 (Git 2.49, 2025-03-14) with\nPatrick's series 'refs: batch refname availability checks' [1] merged\nin.\n\n[1]: https://lore.kernel.org/all/20250217-pks-update-ref-optimization-v1-0-a2b6d87a24af@pks.im/\n\n---\n\nChanges in v6:\n- The documentation for 'git update-ref' didn't repeat the command, giving the intention\n that newlines added were continuation of options rather than alternative invocations.\n- Link to v5: https://lore.kernel.org/all/20250327-245-partially-atomic-ref-updates-v5-0-4db2a3e34404@gmail.com\n\nChanges in v5:\n- Inline the comments around the 'ref_transaction_error'.\n- Use 'strbuf_reset()' wherever possible instead of 'strbuf_setlen(err, 0)'.\n- Use an extra 'conflicting_dirnames' strset in 'refs_verify_refnames_available()' to track\n dirnames which were found to be conflicting, this is to avoid re-reading those dirnames.\n- Add curly braces style mismatch in if..else block.\n- Link to v4: https://lore.kernel.org/r/20250320-245-partially-atomic-ref-updates-v4-0-3dcc1b311dc9@gmail.com\n\nChanges in v4:\n- Rebased on top of 2.49 since there was a long time between the\n previous iteration and we have a new release.\n- Changed the naming to say 'batched' updates instead of 'partial\n transactions'. While we still use the transaction infrastructure\n underneath, the new naming causes less ambiguity.\n- Clean up some of the commit messages.\n- Raise BUG for invalid update index while setting rejections.\n- Fix an incorrect early return.\n- Link to v3: https://lore.kernel.org/r/20250305-245-partially-atomic-ref-updates-v3-0-0c64e3052354@gmail.com\n\nChanges in v3:\n- Changed 'transaction_error' to 'ref_transaction_error' along with the\n error names. Removed 'TRANSACTION_OK' since it can potentially be\n missed instead of simply 'return 0'.\n- Rename 'ref_transaction_set_rejected' to\n 'ref_transaction_maybe_set_rejected' and move logic around error\n checks to within this function.\n- Add a new struct 'ref_transaction_rejections' to track the rejections\n within a transaction. This allows us to only iterate over rejected\n updates.\n- Add a new commit to also support partial transactions within the\n batched F/D checks.\n- Remove NUL delimited outputs in 'git-update-ref(1)'.\n- Remove translations for plumbing outputs.\n- Other small cleanups in the commit message and code.\n\nChanges in v2:\n- Introduce and use structured errors. This consolidates the errors\n and their handling between the ref backends.\n- In the previous version, we skipped over all failures. This include\n system failures such as low memory or IO problems. Let's instead, only\n skip user-oriented failures, such as invalid old OID and so on.\n- Change the rejection function name to `ref_transaction_set_rejected()`.\n- Modify the commit messages and documentation to be a little more\n verbose.\n- Link to v1: https://lore.kernel.org/r/20250207-245-partially-atomic-ref-updates-v1-0-e6a3690ff23a@gmail.com\n\n---\n\nKarthik Nayak (8):\n refs/files: remove redundant check in split_symref_update()\n refs: move duplicate refname update check to generic layer\n refs/files: remove duplicate duplicates check\n refs/reftable: extract code from the transaction preparation\n refs: introduce enum-based transaction error types\n refs: implement batch reference update support\n refs: support rejection in batch updates during F/D checks\n update-ref: add --batch-updates flag for stdin mode\n\n Documentation/git-update-ref.adoc | 14 +-\n builtin/fetch.c | 2 +-\n builtin/update-ref.c | 66 +++-\n refs.c | 171 ++++++++--\n refs.h | 70 +++--\n refs/files-backend.c | 314 ++++++++-----------\n refs/packed-backend.c | 69 ++--\n refs/refs-internal.h | 51 ++-\n refs/reftable-backend.c | 502 +++++++++++++++---------------\n t/t1400-update-ref.sh | 233 ++++++++++++++\n 10 files changed, 969 insertions(+), 523 deletions(-)\n\n---\n\nRange-diff versus v5:\n\n1: cae24142a1 = 1: cae24142a1 refs/files: remove redundant check in split_symref_update()\n2: 239aecdb0f = 2: 239aecdb0f refs: move duplicate refname update check to generic layer\n3: 06404dd350 = 3: 06404dd350 refs/files: remove duplicate duplicates check\n4: a3e645aa37 = 4: a3e645aa37 refs/reftable: extract code from the transaction preparation\n5: 2615bfe78e = 5: 2615bfe78e refs: introduce enum-based transaction error types\n6: d5c1c77b0d = 6: d5c1c77b0d refs: implement batch reference update support\n7: 4bb4902631 = 7: 4bb4902631 refs: support rejection in batch updates during F/D checks\n8: 674630f77c ! 8: ed92beaf18 update-ref: add --batch-updates flag for stdin mode\n @@ Documentation/git-update-ref.adoc: git-update-ref - Update the object name store\n -'git update-ref' [-m ] [--no-deref] (-d [] | [--create-reflog] [] | --stdin [-z])\n +[synopsis]\n +git update-ref [-m ] [--no-deref] -d []\n -+\t [-m ] [--no-deref] [--create-reflog] []\n -+ [-m ] [--no-deref] --stdin [-z] [--batch-updates]\n ++git update-ref [-m ] [--no-deref] [--create-reflog] []\n ++git update-ref [-m ] [--no-deref] --stdin [-z] [--batch-updates]\n \n DESCRIPTION\n -----------\n\n-- \n2.48.1\n\n"},{"id":"515836","messageId":"20250408085120.614893-2-karthik.188@gmail.com","threadId":"62915","inReplyTo":"20250408085120.614893-1-karthik.188@gmail.com","subject":"[PATCH v6 1/8] refs/files: remove redundant check in split_symref_update()","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-08T08:51:05Z","receivedAt":"2025-04-08T08:51:27Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"In `split_symref_update()`, there were two checks for duplicate\nrefnames:\n\n - At the start, `string_list_has_string()` ensures the refname is not\n already in `affected_refnames`, preventing duplicates from being\n added.\n\n - After adding the refname, another check verifies whether the newly\n inserted item has a `util` value.\n\nThe second check is unnecessary because the first one guarantees that\n`string_list_insert()` will never encounter a preexisting entry.\n\nThe `item->util` field is assigned to validate that a rename doesn't\nalready exist in the list. The validation is done after the first check.\nAs this check is removed, clean up the validation and the assignment of\nthis field in `split_head_update()` and `files_transaction_prepare()`.\n\nSigned-off-by: Karthik Nayak \n---\n refs/files-backend.c | 20 +++-----------------\n 1 file changed, 3 insertions(+), 17 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex ff54a4bb7e..15559a09c5 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -2382,7 +2382,6 @@ static int split_head_update(struct ref_update *update,\n \t\t\t struct string_list *affected_refnames,\n \t\t\t struct strbuf *err)\n {\n-\tstruct string_list_item *item;\n \tstruct ref_update *new_update;\n \n \tif ((update->flags & REF_LOG_ONLY) ||\n@@ -2421,8 +2420,7 @@ static int split_head_update(struct ref_update *update,\n \t */\n \tif (strcmp(new_update->refname, \"HEAD\"))\n \t\tBUG(\"%s unexpectedly not 'HEAD'\", new_update->refname);\n-\titem = string_list_insert(affected_refnames, new_update->refname);\n-\titem->util = new_update;\n+\tstring_list_insert(affected_refnames, new_update->refname);\n \n \treturn 0;\n }\n@@ -2441,7 +2439,6 @@ static int split_symref_update(struct ref_update *update,\n \t\t\t struct string_list *affected_refnames,\n \t\t\t struct strbuf *err)\n {\n-\tstruct string_list_item *item;\n \tstruct ref_update *new_update;\n \tunsigned int new_flags;\n \n@@ -2496,11 +2493,7 @@ static int split_symref_update(struct ref_update *update,\n \t * be valid as long as affected_refnames is in use, and NOT\n \t * referent, which might soon be freed by our caller.\n \t */\n-\titem = string_list_insert(affected_refnames, new_update->refname);\n-\tif (item->util)\n-\t\tBUG(\"%s unexpectedly found in affected_refnames\",\n-\t\t new_update->refname);\n-\titem->util = new_update;\n+\tstring_list_insert(affected_refnames, new_update->refname);\n \n \treturn 0;\n }\n@@ -2834,7 +2827,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t */\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n-\t\tstruct string_list_item *item;\n \n \t\tif ((update->flags & REF_IS_PRUNING) &&\n \t\t !(update->flags & REF_NO_DEREF))\n@@ -2843,13 +2835,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t\tif (update->flags & REF_LOG_ONLY)\n \t\t\tcontinue;\n \n-\t\titem = string_list_append(&affected_refnames, update->refname);\n-\t\t/*\n-\t\t * We store a pointer to update in item->util, but at\n-\t\t * the moment we never use the value of this field\n-\t\t * except to check whether it is non-NULL.\n-\t\t */\n-\t\titem->util = update;\n+\t\tstring_list_append(&affected_refnames, update->refname);\n \t}\n \tstring_list_sort(&affected_refnames);\n \tif (ref_update_reject_duplicates(&affected_refnames, err)) {\n-- \n2.48.1\n\n"},{"id":"515837","messageId":"20250408085120.614893-3-karthik.188@gmail.com","threadId":"62915","inReplyTo":"20250408085120.614893-1-karthik.188@gmail.com","subject":"[PATCH v6 2/8] refs: move duplicate refname update check to generic layer","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-08T08:51:06Z","receivedAt":"2025-04-08T08:51:28Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Move the tracking of refnames in `affected_refnames` from individual\nbackends into the generic layer in 'refs.c'. This centralizes the\nduplicate refname detection that was previously handled separately by\neach backend.\n\nMake some changes to accommodate this move:\n\n - Add a `string_list` field `refnames` to `ref_transaction` to contain\n all the references in a transaction. This field is updated whenever\n a new update is added via `ref_transaction_add_update`, so manual\n additions in reference backends are dropped.\n\n - Modify the backends to use this field internally as needed. The\n backends need to check if an update for refname already exists when\n splitting symrefs or adding an update for 'HEAD'.\n\n - In the reftable backend, within `reftable_be_transaction_prepare()`,\n move the `string_list_has_string()` check above\n `ref_transaction_add_update()`. Since `ref_transaction_add_update()`\n automatically adds the refname to `transaction->refnames`,\n performing the check after will always return true, so we perform\n the check before adding the update.\n\nThis helps reduce duplication of functionality between the backends and\nmakes it easier to make changes in a more centralized manner.\n\nSigned-off-by: Karthik Nayak \n---\n refs.c | 17 +++++++++++\n refs/files-backend.c | 67 +++++++++--------------------------------\n refs/packed-backend.c | 25 +--------------\n refs/refs-internal.h | 2 ++\n refs/reftable-backend.c | 54 +++++++++++----------------------\n 5 files changed, 51 insertions(+), 114 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 2ac9d8ebd0..504bf2063e 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1175,6 +1175,7 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,\n \tCALLOC_ARRAY(tr, 1);\n \ttr->ref_store = refs;\n \ttr->flags = flags;\n+\tstring_list_init_dup(&tr->refnames);\n \treturn tr;\n }\n \n@@ -1205,6 +1206,7 @@ void ref_transaction_free(struct ref_transaction *transaction)\n \t\tfree((char *)transaction->updates[i]->old_target);\n \t\tfree(transaction->updates[i]);\n \t}\n+\tstring_list_clear(&transaction->refnames, 0);\n \tfree(transaction->updates);\n \tfree(transaction);\n }\n@@ -1218,6 +1220,7 @@ struct ref_update *ref_transaction_add_update(\n \t\tconst char *committer_info,\n \t\tconst char *msg)\n {\n+\tstruct string_list_item *item;\n \tstruct ref_update *update;\n \n \tif (transaction->state != REF_TRANSACTION_OPEN)\n@@ -1245,6 +1248,16 @@ struct ref_update *ref_transaction_add_update(\n \t\tupdate->msg = normalize_reflog_message(msg);\n \t}\n \n+\t/*\n+\t * This list is generally used by the backends to avoid duplicates.\n+\t * But we do support multiple log updates for a given refname within\n+\t * a single transaction.\n+\t */\n+\tif (!(update->flags & REF_LOG_ONLY)) {\n+\t\titem = string_list_append(&transaction->refnames, refname);\n+\t\titem->util = update;\n+\t}\n+\n \treturn update;\n }\n \n@@ -2405,6 +2418,10 @@ int ref_transaction_prepare(struct ref_transaction *transaction,\n \t\treturn -1;\n \t}\n \n+\tstring_list_sort(&transaction->refnames);\n+\tif (ref_update_reject_duplicates(&transaction->refnames, err))\n+\t\treturn TRANSACTION_GENERIC_ERROR;\n+\n \tret = refs->be->transaction_prepare(refs, transaction, err);\n \tif (ret)\n \t\treturn ret;\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 15559a09c5..58f62ea8a3 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -2378,9 +2378,7 @@ static struct ref_iterator *files_reflog_iterator_begin(struct ref_store *ref_st\n */\n static int split_head_update(struct ref_update *update,\n \t\t\t struct ref_transaction *transaction,\n-\t\t\t const char *head_ref,\n-\t\t\t struct string_list *affected_refnames,\n-\t\t\t struct strbuf *err)\n+\t\t\t const char *head_ref, struct strbuf *err)\n {\n \tstruct ref_update *new_update;\n \n@@ -2398,7 +2396,7 @@ static int split_head_update(struct ref_update *update,\n \t * transaction. This check is O(lg N) in the transaction\n \t * size, but it happens at most once per transaction.\n \t */\n-\tif (string_list_has_string(affected_refnames, \"HEAD\")) {\n+\tif (string_list_has_string(&transaction->refnames, \"HEAD\")) {\n \t\t/* An entry already existed */\n \t\tstrbuf_addf(err,\n \t\t\t \"multiple updates for 'HEAD' (including one \"\n@@ -2420,7 +2418,6 @@ static int split_head_update(struct ref_update *update,\n \t */\n \tif (strcmp(new_update->refname, \"HEAD\"))\n \t\tBUG(\"%s unexpectedly not 'HEAD'\", new_update->refname);\n-\tstring_list_insert(affected_refnames, new_update->refname);\n \n \treturn 0;\n }\n@@ -2436,7 +2433,6 @@ static int split_head_update(struct ref_update *update,\n static int split_symref_update(struct ref_update *update,\n \t\t\t const char *referent,\n \t\t\t struct ref_transaction *transaction,\n-\t\t\t struct string_list *affected_refnames,\n \t\t\t struct strbuf *err)\n {\n \tstruct ref_update *new_update;\n@@ -2448,7 +2444,7 @@ static int split_symref_update(struct ref_update *update,\n \t * size, but it happens at most once per symref in a\n \t * transaction.\n \t */\n-\tif (string_list_has_string(affected_refnames, referent)) {\n+\tif (string_list_has_string(&transaction->refnames, referent)) {\n \t\t/* An entry already exists */\n \t\tstrbuf_addf(err,\n \t\t\t \"multiple updates for '%s' (including one \"\n@@ -2486,15 +2482,6 @@ static int split_symref_update(struct ref_update *update,\n \tupdate->flags |= REF_LOG_ONLY | REF_NO_DEREF;\n \tupdate->flags &= ~REF_HAVE_OLD;\n \n-\t/*\n-\t * Add the referent. This insertion is O(N) in the transaction\n-\t * size, but it happens at most once per symref in a\n-\t * transaction. Make sure to add new_update->refname, which will\n-\t * be valid as long as affected_refnames is in use, and NOT\n-\t * referent, which might soon be freed by our caller.\n-\t */\n-\tstring_list_insert(affected_refnames, new_update->refname);\n-\n \treturn 0;\n }\n \n@@ -2558,7 +2545,6 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\t\t struct ref_transaction *transaction,\n \t\t\t const char *head_ref,\n \t\t\t struct string_list *refnames_to_check,\n-\t\t\t struct string_list *affected_refnames,\n \t\t\t struct strbuf *err)\n {\n \tstruct strbuf referent = STRBUF_INIT;\n@@ -2575,8 +2561,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\tupdate->flags |= REF_DELETING;\n \n \tif (head_ref) {\n-\t\tret = split_head_update(update, transaction, head_ref,\n-\t\t\t\t\taffected_refnames, err);\n+\t\tret = split_head_update(update, transaction, head_ref, err);\n \t\tif (ret)\n \t\t\tgoto out;\n \t}\n@@ -2586,9 +2571,8 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\tlock->count++;\n \t} else {\n \t\tret = lock_raw_ref(refs, update->refname, mustexist,\n-\t\t\t\t refnames_to_check, affected_refnames,\n-\t\t\t\t &lock, &referent,\n-\t\t\t\t &update->type, err);\n+\t\t\t\t refnames_to_check, &transaction->refnames,\n+\t\t\t\t &lock, &referent, &update->type, err);\n \t\tif (ret) {\n \t\t\tchar *reason;\n \n@@ -2642,9 +2626,8 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\t\t * of processing the split-off update, so we\n \t\t\t * don't have to do it here.\n \t\t\t */\n-\t\t\tret = split_symref_update(update,\n-\t\t\t\t\t\t referent.buf, transaction,\n-\t\t\t\t\t\t affected_refnames, err);\n+\t\t\tret = split_symref_update(update, referent.buf,\n+\t\t\t\t\t\t transaction, err);\n \t\t\tif (ret)\n \t\t\t\tgoto out;\n \t\t}\n@@ -2799,7 +2782,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t\t\t \"ref_transaction_prepare\");\n \tsize_t i;\n \tint ret = 0;\n-\tstruct string_list affected_refnames = STRING_LIST_INIT_NODUP;\n \tstruct string_list refnames_to_check = STRING_LIST_INIT_NODUP;\n \tchar *head_ref = NULL;\n \tint head_type;\n@@ -2818,12 +2800,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \ttransaction->backend_data = backend_data;\n \n \t/*\n-\t * Fail if a refname appears more than once in the\n-\t * transaction. (If we end up splitting up any updates using\n-\t * split_symref_update() or split_head_update(), those\n-\t * functions will check that the new updates don't have the\n-\t * same refname as any existing ones.) Also fail if any of the\n-\t * updates use REF_IS_PRUNING without REF_NO_DEREF.\n+\t * Fail if any of the updates use REF_IS_PRUNING without REF_NO_DEREF.\n \t */\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n@@ -2831,16 +2808,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t\tif ((update->flags & REF_IS_PRUNING) &&\n \t\t !(update->flags & REF_NO_DEREF))\n \t\t\tBUG(\"REF_IS_PRUNING set without REF_NO_DEREF\");\n-\n-\t\tif (update->flags & REF_LOG_ONLY)\n-\t\t\tcontinue;\n-\n-\t\tstring_list_append(&affected_refnames, update->refname);\n-\t}\n-\tstring_list_sort(&affected_refnames);\n-\tif (ref_update_reject_duplicates(&affected_refnames, err)) {\n-\t\tret = TRANSACTION_GENERIC_ERROR;\n-\t\tgoto cleanup;\n \t}\n \n \t/*\n@@ -2882,7 +2849,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \n \t\tret = lock_ref_for_update(refs, update, transaction,\n \t\t\t\t\t head_ref, &refnames_to_check,\n-\t\t\t\t\t &affected_refnames, err);\n+\t\t\t\t\t err);\n \t\tif (ret)\n \t\t\tgoto cleanup;\n \n@@ -2929,7 +2896,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t * So instead, we accept the race for now.\n \t */\n \tif (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,\n-\t\t\t\t\t &affected_refnames, NULL, 0, err)) {\n+\t\t\t\t\t &transaction->refnames, NULL, 0, err)) {\n \t\tret = TRANSACTION_NAME_CONFLICT;\n \t\tgoto cleanup;\n \t}\n@@ -2975,7 +2942,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \n cleanup:\n \tfree(head_ref);\n-\tstring_list_clear(&affected_refnames, 0);\n \tstring_list_clear(&refnames_to_check, 0);\n \n \tif (ret)\n@@ -3050,13 +3016,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \tif (transaction->state != REF_TRANSACTION_PREPARED)\n \t\tBUG(\"commit called for transaction that is not prepared\");\n \n-\t/* Fail if a refname appears more than once in the transaction: */\n-\tfor (i = 0; i < transaction->nr; i++)\n-\t\tif (!(transaction->updates[i]->flags & REF_LOG_ONLY))\n-\t\t\tstring_list_append(&affected_refnames,\n-\t\t\t\t\t transaction->updates[i]->refname);\n-\tstring_list_sort(&affected_refnames);\n-\tif (ref_update_reject_duplicates(&affected_refnames, err)) {\n+\tstring_list_sort(&transaction->refnames);\n+\tif (ref_update_reject_duplicates(&transaction->refnames, err)) {\n \t\tret = TRANSACTION_GENERIC_ERROR;\n \t\tgoto cleanup;\n \t}\n@@ -3074,7 +3035,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \t * that we are creating already exists.\n \t */\n \tif (refs_for_each_rawref(&refs->base, ref_present,\n-\t\t\t\t &affected_refnames))\n+\t\t\t\t &transaction->refnames))\n \t\tBUG(\"initial ref transaction called with existing refs\");\n \n \tpacked_transaction = ref_store_transaction_begin(refs->packed_ref_store,\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex f4c82ba2c7..19220d2e99 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1622,8 +1622,6 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n struct packed_transaction_backend_data {\n \t/* True iff the transaction owns the packed-refs lock. */\n \tint own_lock;\n-\n-\tstruct string_list updates;\n };\n \n static void packed_transaction_cleanup(struct packed_ref_store *refs,\n@@ -1632,8 +1630,6 @@ static void packed_transaction_cleanup(struct packed_ref_store *refs,\n \tstruct packed_transaction_backend_data *data = transaction->backend_data;\n \n \tif (data) {\n-\t\tstring_list_clear(&data->updates, 0);\n-\n \t\tif (is_tempfile_active(refs->tempfile))\n \t\t\tdelete_tempfile(&refs->tempfile);\n \n@@ -1658,7 +1654,6 @@ static int packed_transaction_prepare(struct ref_store *ref_store,\n \t\t\tREF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,\n \t\t\t\"ref_transaction_prepare\");\n \tstruct packed_transaction_backend_data *data;\n-\tsize_t i;\n \tint ret = TRANSACTION_GENERIC_ERROR;\n \n \t/*\n@@ -1671,34 +1666,16 @@ static int packed_transaction_prepare(struct ref_store *ref_store,\n \t */\n \n \tCALLOC_ARRAY(data, 1);\n-\tstring_list_init_nodup(&data->updates);\n \n \ttransaction->backend_data = data;\n \n-\t/*\n-\t * Stick the updates in a string list by refname so that we\n-\t * can sort them:\n-\t */\n-\tfor (i = 0; i < transaction->nr; i++) {\n-\t\tstruct ref_update *update = transaction->updates[i];\n-\t\tstruct string_list_item *item =\n-\t\t\tstring_list_append(&data->updates, update->refname);\n-\n-\t\t/* Store a pointer to update in item->util: */\n-\t\titem->util = update;\n-\t}\n-\tstring_list_sort(&data->updates);\n-\n-\tif (ref_update_reject_duplicates(&data->updates, err))\n-\t\tgoto failure;\n-\n \tif (!is_lock_file_locked(&refs->lock)) {\n \t\tif (packed_refs_lock(ref_store, 0, err))\n \t\t\tgoto failure;\n \t\tdata->own_lock = 1;\n \t}\n \n-\tif (write_with_updates(refs, &data->updates, err))\n+\tif (write_with_updates(refs, &transaction->refnames, err))\n \t\tgoto failure;\n \n \ttransaction->state = REF_TRANSACTION_PREPARED;\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex e5862757a7..92db793026 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -3,6 +3,7 @@\n \n #include \"refs.h\"\n #include \"iterator.h\"\n+#include \"string-list.h\"\n \n struct fsck_options;\n struct ref_transaction;\n@@ -198,6 +199,7 @@ enum ref_transaction_state {\n struct ref_transaction {\n \tstruct ref_store *ref_store;\n \tstruct ref_update **updates;\n+\tstruct string_list refnames;\n \tsize_t alloc;\n \tsize_t nr;\n \tenum ref_transaction_state state;\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex ae434cd248..a92c9a2f4f 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -1076,7 +1076,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \tstruct reftable_ref_store *refs =\n \t\treftable_be_downcast(ref_store, REF_STORE_WRITE|REF_STORE_MAIN, \"ref_transaction_prepare\");\n \tstruct strbuf referent = STRBUF_INIT, head_referent = STRBUF_INIT;\n-\tstruct string_list affected_refnames = STRING_LIST_INIT_NODUP;\n \tstruct string_list refnames_to_check = STRING_LIST_INIT_NODUP;\n \tstruct reftable_transaction_data *tx_data = NULL;\n \tstruct reftable_backend *be;\n@@ -1101,10 +1100,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\t\t\t transaction->updates[i], err);\n \t\tif (ret)\n \t\t\tgoto done;\n-\n-\t\tif (!(transaction->updates[i]->flags & REF_LOG_ONLY))\n-\t\t\tstring_list_append(&affected_refnames,\n-\t\t\t\t\t transaction->updates[i]->refname);\n \t}\n \n \t/*\n@@ -1116,17 +1111,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\ttx_data->args[i].updates_alloc = tx_data->args[i].updates_expected;\n \t}\n \n-\t/*\n-\t * Fail if a refname appears more than once in the transaction.\n-\t * This code is taken from the files backend and is a good candidate to\n-\t * be moved into the generic layer.\n-\t */\n-\tstring_list_sort(&affected_refnames);\n-\tif (ref_update_reject_duplicates(&affected_refnames, err)) {\n-\t\tret = TRANSACTION_GENERIC_ERROR;\n-\t\tgoto done;\n-\t}\n-\n \t/*\n \t * TODO: it's dubious whether we should reload the stack that \"HEAD\"\n \t * belongs to or not. In theory, it may happen that we only modify\n@@ -1194,14 +1178,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t !(u->flags & REF_LOG_ONLY) &&\n \t\t !(u->flags & REF_UPDATE_VIA_HEAD) &&\n \t\t !strcmp(rewritten_ref, head_referent.buf)) {\n-\t\t\tstruct ref_update *new_update;\n-\n \t\t\t/*\n \t\t\t * First make sure that HEAD is not already in the\n \t\t\t * transaction. This check is O(lg N) in the transaction\n \t\t\t * size, but it happens at most once per transaction.\n \t\t\t */\n-\t\t\tif (string_list_has_string(&affected_refnames, \"HEAD\")) {\n+\t\t\tif (string_list_has_string(&transaction->refnames, \"HEAD\")) {\n \t\t\t\t/* An entry already existed */\n \t\t\t\tstrbuf_addf(err,\n \t\t\t\t\t _(\"multiple updates for 'HEAD' (including one \"\n@@ -1211,12 +1193,11 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\tgoto done;\n \t\t\t}\n \n-\t\t\tnew_update = ref_transaction_add_update(\n-\t\t\t\t\ttransaction, \"HEAD\",\n-\t\t\t\t\tu->flags | REF_LOG_ONLY | REF_NO_DEREF,\n-\t\t\t\t\t&u->new_oid, &u->old_oid, NULL, NULL, NULL,\n-\t\t\t\t\tu->msg);\n-\t\t\tstring_list_insert(&affected_refnames, new_update->refname);\n+\t\t\tref_transaction_add_update(\n+\t\t\t\ttransaction, \"HEAD\",\n+\t\t\t\tu->flags | REF_LOG_ONLY | REF_NO_DEREF,\n+\t\t\t\t&u->new_oid, &u->old_oid, NULL, NULL, NULL,\n+\t\t\t\tu->msg);\n \t\t}\n \n \t\tret = reftable_backend_read_ref(be, rewritten_ref,\n@@ -1281,6 +1262,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\tif (!strcmp(rewritten_ref, \"HEAD\"))\n \t\t\t\t\tnew_flags |= REF_UPDATE_VIA_HEAD;\n \n+\t\t\t\tif (string_list_has_string(&transaction->refnames, referent.buf)) {\n+\t\t\t\t\tstrbuf_addf(err,\n+\t\t\t\t\t\t _(\"multiple updates for '%s' (including one \"\n+\t\t\t\t\t\t \"via symref '%s') are not allowed\"),\n+\t\t\t\t\t\t referent.buf, u->refname);\n+\t\t\t\t\tret = TRANSACTION_NAME_CONFLICT;\n+\t\t\t\t\tgoto done;\n+\t\t\t\t}\n+\n \t\t\t\t/*\n \t\t\t\t * If we are updating a symref (eg. HEAD), we should also\n \t\t\t\t * update the branch that the symref points to.\n@@ -1305,16 +1295,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\t */\n \t\t\t\tu->flags |= REF_LOG_ONLY | REF_NO_DEREF;\n \t\t\t\tu->flags &= ~REF_HAVE_OLD;\n-\n-\t\t\t\tif (string_list_has_string(&affected_refnames, new_update->refname)) {\n-\t\t\t\t\tstrbuf_addf(err,\n-\t\t\t\t\t\t _(\"multiple updates for '%s' (including one \"\n-\t\t\t\t\t\t \"via symref '%s') are not allowed\"),\n-\t\t\t\t\t\t referent.buf, u->refname);\n-\t\t\t\t\tret = TRANSACTION_NAME_CONFLICT;\n-\t\t\t\t\tgoto done;\n-\t\t\t\t}\n-\t\t\t\tstring_list_insert(&affected_refnames, new_update->refname);\n \t\t\t}\n \t\t}\n \n@@ -1383,7 +1363,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t}\n \t}\n \n-\tret = refs_verify_refnames_available(ref_store, &refnames_to_check, &affected_refnames, NULL,\n+\tret = refs_verify_refnames_available(ref_store, &refnames_to_check,\n+\t\t\t\t\t &transaction->refnames, NULL,\n \t\t\t\t\t transaction->flags & REF_TRANSACTION_FLAG_INITIAL,\n \t\t\t\t\t err);\n \tif (ret < 0)\n@@ -1401,7 +1382,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\tstrbuf_addf(err, _(\"reftable: transaction prepare: %s\"),\n \t\t\t\t reftable_error_str(ret));\n \t}\n-\tstring_list_clear(&affected_refnames, 0);\n \tstrbuf_release(&referent);\n \tstrbuf_release(&head_referent);\n \tstring_list_clear(&refnames_to_check, 0);\n-- \n2.48.1\n\n"},{"id":"515838","messageId":"20250408085120.614893-4-karthik.188@gmail.com","threadId":"62915","inReplyTo":"20250408085120.614893-1-karthik.188@gmail.com","subject":"[PATCH v6 3/8] refs/files: remove duplicate duplicates check","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-08T08:51:07Z","receivedAt":"2025-04-08T08:51:29Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Within the files reference backend's transaction's 'finish' phase, a\nverification step is currently performed wherein the refnames list is\nsorted and examined for multiple updates targeting the same refname.\n\nIt has been observed that this verification is redundant, as an\nidentical check is already executed during the transaction's 'prepare'\nstage. Since the refnames list remains unmodified following the\n'prepare' stage, this secondary verification can be safely eliminated.\n\nThe duplicate check has been removed accordingly, and the\n`ref_update_reject_duplicates()` function has been marked as static, as\nits usage is now confined to 'refs.c'.\n\nSigned-off-by: Karthik Nayak \n---\n refs.c | 9 +++++++--\n refs/files-backend.c | 6 ------\n refs/refs-internal.h | 8 --------\n 3 files changed, 7 insertions(+), 16 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 504bf2063e..61bed9672a 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2303,8 +2303,13 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,\n \treturn ret;\n }\n \n-int ref_update_reject_duplicates(struct string_list *refnames,\n-\t\t\t\t struct strbuf *err)\n+/*\n+ * Write an error to `err` and return a nonzero value iff the same\n+ * refname appears multiple times in `refnames`. `refnames` must be\n+ * sorted on entry to this function.\n+ */\n+static int ref_update_reject_duplicates(struct string_list *refnames,\n+\t\t\t\t\tstruct strbuf *err)\n {\n \tsize_t i, n = refnames->nr;\n \ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 58f62ea8a3..ea023a59fc 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3016,12 +3016,6 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \tif (transaction->state != REF_TRANSACTION_PREPARED)\n \t\tBUG(\"commit called for transaction that is not prepared\");\n \n-\tstring_list_sort(&transaction->refnames);\n-\tif (ref_update_reject_duplicates(&transaction->refnames, err)) {\n-\t\tret = TRANSACTION_GENERIC_ERROR;\n-\t\tgoto cleanup;\n-\t}\n-\n \t/*\n \t * It's really undefined to call this function in an active\n \t * repository or when there are existing references: we are\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 92db793026..6d3770d0cc 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -142,14 +142,6 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t struct object_id *oid, struct strbuf *referent,\n \t\t unsigned int *type, int *failure_errno);\n \n-/*\n- * Write an error to `err` and return a nonzero value iff the same\n- * refname appears multiple times in `refnames`. `refnames` must be\n- * sorted on entry to this function.\n- */\n-int ref_update_reject_duplicates(struct string_list *refnames,\n-\t\t\t\t struct strbuf *err);\n-\n /*\n * Add a ref_update with the specified properties to transaction, and\n * return a pointer to the new object. This function does not verify\n-- \n2.48.1\n\n"},{"id":"515839","messageId":"20250408085120.614893-5-karthik.188@gmail.com","threadId":"62915","inReplyTo":"20250408085120.614893-1-karthik.188@gmail.com","subject":"[PATCH v6 4/8] refs/reftable: extract code from the transaction preparation","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-08T08:51:08Z","receivedAt":"2025-04-08T08:51:30Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Extract the core logic for preparing individual reference updates from\n`reftable_be_transaction_prepare()` into `prepare_single_update()`. This\ndedicated function now handles all validation and preparation steps for\neach reference update in the transaction, including object ID\nverification, HEAD reference handling, and symref processing.\n\nThe refactoring consolidates all reference update validation into a\nsingle logical block, which improves code maintainability and\nreadability. More importantly, this restructuring lays the groundwork\nfor implementing batched reference update support in the reftable\nbackend, which will be introduced in a followup commit.\n\nNo functional changes are included in this commit - it is purely a code\nreorganization to support future enhancements.\n\nSigned-off-by: Karthik Nayak \n---\n refs/reftable-backend.c | 463 ++++++++++++++++++++--------------------\n 1 file changed, 237 insertions(+), 226 deletions(-)\n\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex a92c9a2f4f..786df11a03 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -1069,6 +1069,239 @@ static int queue_transaction_update(struct reftable_ref_store *refs,\n \treturn 0;\n }\n \n+static int prepare_single_update(struct reftable_ref_store *refs,\n+\t\t\t\t struct reftable_transaction_data *tx_data,\n+\t\t\t\t struct ref_transaction *transaction,\n+\t\t\t\t struct reftable_backend *be,\n+\t\t\t\t struct ref_update *u,\n+\t\t\t\t struct string_list *refnames_to_check,\n+\t\t\t\t unsigned int head_type,\n+\t\t\t\t struct strbuf *head_referent,\n+\t\t\t\t struct strbuf *referent,\n+\t\t\t\t struct strbuf *err)\n+{\n+\tstruct object_id current_oid = {0};\n+\tconst char *rewritten_ref;\n+\tint ret = 0;\n+\n+\t/*\n+\t * There is no need to reload the respective backends here as\n+\t * we have already reloaded them when preparing the transaction\n+\t * update. And given that the stacks have been locked there\n+\t * shouldn't have been any concurrent modifications of the\n+\t * stack.\n+\t */\n+\tret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\t/* Verify that the new object ID is valid. */\n+\tif ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&\n+\t !(u->flags & REF_SKIP_OID_VERIFICATION) &&\n+\t !(u->flags & REF_LOG_ONLY)) {\n+\t\tstruct object *o = parse_object(refs->base.repo, &u->new_oid);\n+\t\tif (!o) {\n+\t\t\tstrbuf_addf(err,\n+\t\t\t\t _(\"trying to write ref '%s' with nonexistent object %s\"),\n+\t\t\t\t u->refname, oid_to_hex(&u->new_oid));\n+\t\t\treturn -1;\n+\t\t}\n+\n+\t\tif (o->type != OBJ_COMMIT && is_branch(u->refname)) {\n+\t\t\tstrbuf_addf(err, _(\"trying to write non-commit object %s to branch '%s'\"),\n+\t\t\t\t oid_to_hex(&u->new_oid), u->refname);\n+\t\t\treturn -1;\n+\t\t}\n+\t}\n+\n+\t/*\n+\t * When we update the reference that HEAD points to we enqueue\n+\t * a second log-only update for HEAD so that its reflog is\n+\t * updated accordingly.\n+\t */\n+\tif (head_type == REF_ISSYMREF &&\n+\t !(u->flags & REF_LOG_ONLY) &&\n+\t !(u->flags & REF_UPDATE_VIA_HEAD) &&\n+\t !strcmp(rewritten_ref, head_referent->buf)) {\n+\t\t/*\n+\t\t * First make sure that HEAD is not already in the\n+\t\t * transaction. This check is O(lg N) in the transaction\n+\t\t * size, but it happens at most once per transaction.\n+\t\t */\n+\t\tif (string_list_has_string(&transaction->refnames, \"HEAD\")) {\n+\t\t\t/* An entry already existed */\n+\t\t\tstrbuf_addf(err,\n+\t\t\t\t _(\"multiple updates for 'HEAD' (including one \"\n+\t\t\t\t \"via its referent '%s') are not allowed\"),\n+\t\t\t\t u->refname);\n+\t\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t\t}\n+\n+\t\tref_transaction_add_update(\n+\t\t\ttransaction, \"HEAD\",\n+\t\t\tu->flags | REF_LOG_ONLY | REF_NO_DEREF,\n+\t\t\t&u->new_oid, &u->old_oid, NULL, NULL, NULL,\n+\t\t\tu->msg);\n+\t}\n+\n+\tret = reftable_backend_read_ref(be, rewritten_ref,\n+\t\t\t\t\t¤t_oid, referent, &u->type);\n+\tif (ret < 0)\n+\t\treturn ret;\n+\tif (ret > 0 && !ref_update_expects_existing_old_ref(u)) {\n+\t\t/*\n+\t\t * The reference does not exist, and we either have no\n+\t\t * old object ID or expect the reference to not exist.\n+\t\t * We can thus skip below safety checks as well as the\n+\t\t * symref splitting. But we do want to verify that\n+\t\t * there is no conflicting reference here so that we\n+\t\t * can output a proper error message instead of failing\n+\t\t * at a later point.\n+\t\t */\n+\t\tstring_list_append(refnames_to_check, u->refname);\n+\n+\t\t/*\n+\t\t * There is no need to write the reference deletion\n+\t\t * when the reference in question doesn't exist.\n+\t\t */\n+\t\tif ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {\n+\t\t\tret = queue_transaction_update(refs, tx_data, u,\n+\t\t\t\t\t\t ¤t_oid, err);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\t\t}\n+\n+\t\treturn 0;\n+\t}\n+\tif (ret > 0) {\n+\t\t/* The reference does not exist, but we expected it to. */\n+\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n+\n+\n+\t\t\t\t \"unable to resolve reference '%s'\"),\n+\t\t\t ref_update_original_update_refname(u), u->refname);\n+\t\treturn -1;\n+\t}\n+\n+\tif (u->type & REF_ISSYMREF) {\n+\t\t/*\n+\t\t * The reftable stack is locked at this point already,\n+\t\t * so it is safe to call `refs_resolve_ref_unsafe()`\n+\t\t * here without causing races.\n+\t\t */\n+\t\tconst char *resolved = refs_resolve_ref_unsafe(&refs->base, u->refname, 0,\n+\t\t\t\t\t\t\t ¤t_oid, NULL);\n+\n+\t\tif (u->flags & REF_NO_DEREF) {\n+\t\t\tif (u->flags & REF_HAVE_OLD && !resolved) {\n+\t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n+\t\t\t\t\t\t \"error reading reference\"), u->refname);\n+\t\t\t\treturn -1;\n+\t\t\t}\n+\t\t} else {\n+\t\t\tstruct ref_update *new_update;\n+\t\t\tint new_flags;\n+\n+\t\t\tnew_flags = u->flags;\n+\t\t\tif (!strcmp(rewritten_ref, \"HEAD\"))\n+\t\t\t\tnew_flags |= REF_UPDATE_VIA_HEAD;\n+\n+\t\t\tif (string_list_has_string(&transaction->refnames, referent->buf)) {\n+\t\t\t\tstrbuf_addf(err,\n+\t\t\t\t\t _(\"multiple updates for '%s' (including one \"\n+\t\t\t\t\t \"via symref '%s') are not allowed\"),\n+\t\t\t\t\t referent->buf, u->refname);\n+\t\t\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t\t\t}\n+\n+\t\t\t/*\n+\t\t\t * If we are updating a symref (eg. HEAD), we should also\n+\t\t\t * update the branch that the symref points to.\n+\t\t\t *\n+\t\t\t * This is generic functionality, and would be better\n+\t\t\t * done in refs.c, but the current implementation is\n+\t\t\t * intertwined with the locking in files-backend.c.\n+\t\t\t */\n+\t\t\tnew_update = ref_transaction_add_update(\n+\t\t\t\ttransaction, referent->buf, new_flags,\n+\t\t\t\tu->new_target ? NULL : &u->new_oid,\n+\t\t\t\tu->old_target ? NULL : &u->old_oid,\n+\t\t\t\tu->new_target, u->old_target,\n+\t\t\t\tu->committer_info, u->msg);\n+\n+\t\t\tnew_update->parent_update = u;\n+\n+\t\t\t/*\n+\t\t\t * Change the symbolic ref update to log only. Also, it\n+\t\t\t * doesn't need to check its old OID value, as that will be\n+\t\t\t * done when new_update is processed.\n+\t\t\t */\n+\t\t\tu->flags |= REF_LOG_ONLY | REF_NO_DEREF;\n+\t\t\tu->flags &= ~REF_HAVE_OLD;\n+\t\t}\n+\t}\n+\n+\t/*\n+\t * Verify that the old object matches our expectations. Note\n+\t * that the error messages here do not make a lot of sense in\n+\t * the context of the reftable backend as we never lock\n+\t * individual refs. But the error messages match what the files\n+\t * backend returns, which keeps our tests happy.\n+\t */\n+\tif (u->old_target) {\n+\t\tif (!(u->type & REF_ISSYMREF)) {\n+\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n+\t\t\t\t\t \"expected symref with target '%s': \"\n+\t\t\t\t\t \"but is a regular ref\"),\n+\t\t\t\t ref_update_original_update_refname(u),\n+\t\t\t\t u->old_target);\n+\t\t\treturn -1;\n+\t\t}\n+\n+\t\tif (ref_update_check_old_target(referent->buf, u, err)) {\n+\t\t\treturn -1;\n+\t\t}\n+\t} else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {\n+\t\tif (is_null_oid(&u->old_oid)) {\n+\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n+\t\t\t\t\t \"reference already exists\"),\n+\t\t\t\t ref_update_original_update_refname(u));\n+\t\t\treturn TRANSACTION_CREATE_EXISTS;\n+\t\t}\n+\t\telse if (is_null_oid(¤t_oid))\n+\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n+\t\t\t\t\t \"reference is missing but expected %s\"),\n+\t\t\t\t ref_update_original_update_refname(u),\n+\t\t\t\t oid_to_hex(&u->old_oid));\n+\t\telse\n+\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n+\t\t\t\t\t \"is at %s but expected %s\"),\n+\t\t\t\t ref_update_original_update_refname(u),\n+\t\t\t\t oid_to_hex(¤t_oid),\n+\t\t\t\t oid_to_hex(&u->old_oid));\n+\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t}\n+\n+\t/*\n+\t * If all of the following conditions are true:\n+\t *\n+\t * - We're not about to write a symref.\n+\t * - We're not about to write a log-only entry.\n+\t * - Old and new object ID are different.\n+\t *\n+\t * Then we're essentially doing a no-op update that can be\n+\t * skipped. This is not only for the sake of efficiency, but\n+\t * also skips writing unneeded reflog entries.\n+\t */\n+\tif ((u->type & REF_ISSYMREF) ||\n+\t (u->flags & REF_LOG_ONLY) ||\n+\t (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid)))\n+\t\treturn queue_transaction_update(refs, tx_data, u,\n+\t\t\t\t\t ¤t_oid, err);\n+\n+\treturn 0;\n+}\n+\n static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\t\t struct ref_transaction *transaction,\n \t\t\t\t\t struct strbuf *err)\n@@ -1133,234 +1366,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \tret = 0;\n \n \tfor (i = 0; i < transaction->nr; i++) {\n-\t\tstruct ref_update *u = transaction->updates[i];\n-\t\tstruct object_id current_oid = {0};\n-\t\tconst char *rewritten_ref;\n-\n-\t\t/*\n-\t\t * There is no need to reload the respective backends here as\n-\t\t * we have already reloaded them when preparing the transaction\n-\t\t * update. And given that the stacks have been locked there\n-\t\t * shouldn't have been any concurrent modifications of the\n-\t\t * stack.\n-\t\t */\n-\t\tret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);\n+\t\tret = prepare_single_update(refs, tx_data, transaction, be,\n+\t\t\t\t\t transaction->updates[i],\n+\t\t\t\t\t &refnames_to_check, head_type,\n+\t\t\t\t\t &head_referent, &referent, err);\n \t\tif (ret)\n \t\t\tgoto done;\n-\n-\t\t/* Verify that the new object ID is valid. */\n-\t\tif ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&\n-\t\t !(u->flags & REF_SKIP_OID_VERIFICATION) &&\n-\t\t !(u->flags & REF_LOG_ONLY)) {\n-\t\t\tstruct object *o = parse_object(refs->base.repo, &u->new_oid);\n-\t\t\tif (!o) {\n-\t\t\t\tstrbuf_addf(err,\n-\t\t\t\t\t _(\"trying to write ref '%s' with nonexistent object %s\"),\n-\t\t\t\t\t u->refname, oid_to_hex(&u->new_oid));\n-\t\t\t\tret = -1;\n-\t\t\t\tgoto done;\n-\t\t\t}\n-\n-\t\t\tif (o->type != OBJ_COMMIT && is_branch(u->refname)) {\n-\t\t\t\tstrbuf_addf(err, _(\"trying to write non-commit object %s to branch '%s'\"),\n-\t\t\t\t\t oid_to_hex(&u->new_oid), u->refname);\n-\t\t\t\tret = -1;\n-\t\t\t\tgoto done;\n-\t\t\t}\n-\t\t}\n-\n-\t\t/*\n-\t\t * When we update the reference that HEAD points to we enqueue\n-\t\t * a second log-only update for HEAD so that its reflog is\n-\t\t * updated accordingly.\n-\t\t */\n-\t\tif (head_type == REF_ISSYMREF &&\n-\t\t !(u->flags & REF_LOG_ONLY) &&\n-\t\t !(u->flags & REF_UPDATE_VIA_HEAD) &&\n-\t\t !strcmp(rewritten_ref, head_referent.buf)) {\n-\t\t\t/*\n-\t\t\t * First make sure that HEAD is not already in the\n-\t\t\t * transaction. This check is O(lg N) in the transaction\n-\t\t\t * size, but it happens at most once per transaction.\n-\t\t\t */\n-\t\t\tif (string_list_has_string(&transaction->refnames, \"HEAD\")) {\n-\t\t\t\t/* An entry already existed */\n-\t\t\t\tstrbuf_addf(err,\n-\t\t\t\t\t _(\"multiple updates for 'HEAD' (including one \"\n-\t\t\t\t\t \"via its referent '%s') are not allowed\"),\n-\t\t\t\t\t u->refname);\n-\t\t\t\tret = TRANSACTION_NAME_CONFLICT;\n-\t\t\t\tgoto done;\n-\t\t\t}\n-\n-\t\t\tref_transaction_add_update(\n-\t\t\t\ttransaction, \"HEAD\",\n-\t\t\t\tu->flags | REF_LOG_ONLY | REF_NO_DEREF,\n-\t\t\t\t&u->new_oid, &u->old_oid, NULL, NULL, NULL,\n-\t\t\t\tu->msg);\n-\t\t}\n-\n-\t\tret = reftable_backend_read_ref(be, rewritten_ref,\n-\t\t\t\t\t\t¤t_oid, &referent, &u->type);\n-\t\tif (ret < 0)\n-\t\t\tgoto done;\n-\t\tif (ret > 0 && !ref_update_expects_existing_old_ref(u)) {\n-\t\t\t/*\n-\t\t\t * The reference does not exist, and we either have no\n-\t\t\t * old object ID or expect the reference to not exist.\n-\t\t\t * We can thus skip below safety checks as well as the\n-\t\t\t * symref splitting. But we do want to verify that\n-\t\t\t * there is no conflicting reference here so that we\n-\t\t\t * can output a proper error message instead of failing\n-\t\t\t * at a later point.\n-\t\t\t */\n-\t\t\tstring_list_append(&refnames_to_check, u->refname);\n-\n-\t\t\t/*\n-\t\t\t * There is no need to write the reference deletion\n-\t\t\t * when the reference in question doesn't exist.\n-\t\t\t */\n-\t\t\t if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {\n-\t\t\t\t ret = queue_transaction_update(refs, tx_data, u,\n-\t\t\t\t\t\t\t\t¤t_oid, err);\n-\t\t\t\t if (ret)\n-\t\t\t\t\t goto done;\n-\t\t\t }\n-\n-\t\t\tcontinue;\n-\t\t}\n-\t\tif (ret > 0) {\n-\t\t\t/* The reference does not exist, but we expected it to. */\n-\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n-\t\t\t\t \"unable to resolve reference '%s'\"),\n-\t\t\t\t ref_update_original_update_refname(u), u->refname);\n-\t\t\tret = -1;\n-\t\t\tgoto done;\n-\t\t}\n-\n-\t\tif (u->type & REF_ISSYMREF) {\n-\t\t\t/*\n-\t\t\t * The reftable stack is locked at this point already,\n-\t\t\t * so it is safe to call `refs_resolve_ref_unsafe()`\n-\t\t\t * here without causing races.\n-\t\t\t */\n-\t\t\tconst char *resolved = refs_resolve_ref_unsafe(&refs->base, u->refname, 0,\n-\t\t\t\t\t\t\t\t ¤t_oid, NULL);\n-\n-\t\t\tif (u->flags & REF_NO_DEREF) {\n-\t\t\t\tif (u->flags & REF_HAVE_OLD && !resolved) {\n-\t\t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n-\t\t\t\t\t\t \"error reading reference\"), u->refname);\n-\t\t\t\t\tret = -1;\n-\t\t\t\t\tgoto done;\n-\t\t\t\t}\n-\t\t\t} else {\n-\t\t\t\tstruct ref_update *new_update;\n-\t\t\t\tint new_flags;\n-\n-\t\t\t\tnew_flags = u->flags;\n-\t\t\t\tif (!strcmp(rewritten_ref, \"HEAD\"))\n-\t\t\t\t\tnew_flags |= REF_UPDATE_VIA_HEAD;\n-\n-\t\t\t\tif (string_list_has_string(&transaction->refnames, referent.buf)) {\n-\t\t\t\t\tstrbuf_addf(err,\n-\t\t\t\t\t\t _(\"multiple updates for '%s' (including one \"\n-\t\t\t\t\t\t \"via symref '%s') are not allowed\"),\n-\t\t\t\t\t\t referent.buf, u->refname);\n-\t\t\t\t\tret = TRANSACTION_NAME_CONFLICT;\n-\t\t\t\t\tgoto done;\n-\t\t\t\t}\n-\n-\t\t\t\t/*\n-\t\t\t\t * If we are updating a symref (eg. HEAD), we should also\n-\t\t\t\t * update the branch that the symref points to.\n-\t\t\t\t *\n-\t\t\t\t * This is generic functionality, and would be better\n-\t\t\t\t * done in refs.c, but the current implementation is\n-\t\t\t\t * intertwined with the locking in files-backend.c.\n-\t\t\t\t */\n-\t\t\t\tnew_update = ref_transaction_add_update(\n-\t\t\t\t\ttransaction, referent.buf, new_flags,\n-\t\t\t\t\tu->new_target ? NULL : &u->new_oid,\n-\t\t\t\t\tu->old_target ? NULL : &u->old_oid,\n-\t\t\t\t\tu->new_target, u->old_target,\n-\t\t\t\t\tu->committer_info, u->msg);\n-\n-\t\t\t\tnew_update->parent_update = u;\n-\n-\t\t\t\t/*\n-\t\t\t\t * Change the symbolic ref update to log only. Also, it\n-\t\t\t\t * doesn't need to check its old OID value, as that will be\n-\t\t\t\t * done when new_update is processed.\n-\t\t\t\t */\n-\t\t\t\tu->flags |= REF_LOG_ONLY | REF_NO_DEREF;\n-\t\t\t\tu->flags &= ~REF_HAVE_OLD;\n-\t\t\t}\n-\t\t}\n-\n-\t\t/*\n-\t\t * Verify that the old object matches our expectations. Note\n-\t\t * that the error messages here do not make a lot of sense in\n-\t\t * the context of the reftable backend as we never lock\n-\t\t * individual refs. But the error messages match what the files\n-\t\t * backend returns, which keeps our tests happy.\n-\t\t */\n-\t\tif (u->old_target) {\n-\t\t\tif (!(u->type & REF_ISSYMREF)) {\n-\t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n-\t\t\t\t\t \"expected symref with target '%s': \"\n-\t\t\t\t\t \"but is a regular ref\"),\n-\t\t\t\t\t ref_update_original_update_refname(u),\n-\t\t\t\t\t u->old_target);\n-\t\t\t\tret = -1;\n-\t\t\t\tgoto done;\n-\t\t\t}\n-\n-\t\t\tif (ref_update_check_old_target(referent.buf, u, err)) {\n-\t\t\t\tret = -1;\n-\t\t\t\tgoto done;\n-\t\t\t}\n-\t\t} else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {\n-\t\t\tret = TRANSACTION_NAME_CONFLICT;\n-\t\t\tif (is_null_oid(&u->old_oid)) {\n-\t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n-\t\t\t\t\t\t \"reference already exists\"),\n-\t\t\t\t\t ref_update_original_update_refname(u));\n-\t\t\t\tret = TRANSACTION_CREATE_EXISTS;\n-\t\t\t}\n-\t\t\telse if (is_null_oid(¤t_oid))\n-\t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n-\t\t\t\t\t\t \"reference is missing but expected %s\"),\n-\t\t\t\t\t ref_update_original_update_refname(u),\n-\t\t\t\t\t oid_to_hex(&u->old_oid));\n-\t\t\telse\n-\t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n-\t\t\t\t\t\t \"is at %s but expected %s\"),\n-\t\t\t\t\t ref_update_original_update_refname(u),\n-\t\t\t\t\t oid_to_hex(¤t_oid),\n-\t\t\t\t\t oid_to_hex(&u->old_oid));\n-\t\t\tgoto done;\n-\t\t}\n-\n-\t\t/*\n-\t\t * If all of the following conditions are true:\n-\t\t *\n-\t\t * - We're not about to write a symref.\n-\t\t * - We're not about to write a log-only entry.\n-\t\t * - Old and new object ID are different.\n-\t\t *\n-\t\t * Then we're essentially doing a no-op update that can be\n-\t\t * skipped. This is not only for the sake of efficiency, but\n-\t\t * also skips writing unneeded reflog entries.\n-\t\t */\n-\t\tif ((u->type & REF_ISSYMREF) ||\n-\t\t (u->flags & REF_LOG_ONLY) ||\n-\t\t (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid))) {\n-\t\t\tret = queue_transaction_update(refs, tx_data, u,\n-\t\t\t\t\t\t ¤t_oid, err);\n-\t\t\tif (ret)\n-\t\t\t\tgoto done;\n-\t\t}\n \t}\n \n \tret = refs_verify_refnames_available(ref_store, &refnames_to_check,\n-- \n2.48.1\n\n"},{"id":"515841","messageId":"20250408085120.614893-6-karthik.188@gmail.com","threadId":"62915","inReplyTo":"20250408085120.614893-1-karthik.188@gmail.com","subject":"[PATCH v6 5/8] refs: introduce enum-based transaction error types","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-08T08:51:09Z","receivedAt":"2025-04-08T08:51:31Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Replace preprocessor-defined transaction errors with a strongly-typed\nenum `ref_transaction_error`. This change:\n\n - Improves type safety and function signature clarity.\n - Makes error handling more explicit and discoverable.\n - Maintains existing error cases, while adding new error cases for\n common scenarios.\n\nThis refactoring paves the way for more comprehensive error handling\nwhich we will utilize in the upcoming commits to add batch reference\nupdate support.\n\nSigned-off-by: Karthik Nayak \n---\n builtin/fetch.c | 2 +-\n refs.c | 49 +++++-----\n refs.h | 48 ++++++----\n refs/files-backend.c | 202 ++++++++++++++++++++--------------------\n refs/packed-backend.c | 23 +++--\n refs/refs-internal.h | 5 +-\n refs/reftable-backend.c | 64 ++++++-------\n 7 files changed, 207 insertions(+), 186 deletions(-)\n\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex 95fd0018b9..7615c17faf 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -687,7 +687,7 @@ static int s_update_ref(const char *action,\n \t\tswitch (ref_transaction_commit(our_transaction, &err)) {\n \t\tcase 0:\n \t\t\tbreak;\n-\t\tcase TRANSACTION_NAME_CONFLICT:\n+\t\tcase REF_TRANSACTION_ERROR_NAME_CONFLICT:\n \t\t\tret = STORE_REF_ERROR_DF_CONFLICT;\n \t\t\tgoto out;\n \t\tdefault:\ndiff --git a/refs.c b/refs.c\nindex 61bed9672a..3d0b53d56e 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2271,7 +2271,7 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,\n \t\t\t\t\t REF_NO_DEREF, logmsg, &err))\n \t\t\tgoto error_return;\n \t\tprepret = ref_transaction_prepare(transaction, &err);\n-\t\tif (prepret && prepret != TRANSACTION_CREATE_EXISTS)\n+\t\tif (prepret && prepret != REF_TRANSACTION_ERROR_CREATE_EXISTS)\n \t\t\tgoto error_return;\n \t} else {\n \t\tif (ref_transaction_update(transaction, ref, NULL, NULL,\n@@ -2289,7 +2289,7 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,\n \t\t}\n \t}\n \n-\tif (prepret == TRANSACTION_CREATE_EXISTS)\n+\tif (prepret == REF_TRANSACTION_ERROR_CREATE_EXISTS)\n \t\tgoto cleanup;\n \n \tif (ref_transaction_commit(transaction, &err))\n@@ -2425,7 +2425,7 @@ int ref_transaction_prepare(struct ref_transaction *transaction,\n \n \tstring_list_sort(&transaction->refnames);\n \tif (ref_update_reject_duplicates(&transaction->refnames, err))\n-\t\treturn TRANSACTION_GENERIC_ERROR;\n+\t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \n \tret = refs->be->transaction_prepare(refs, transaction, err);\n \tif (ret)\n@@ -2497,19 +2497,19 @@ int ref_transaction_commit(struct ref_transaction *transaction,\n \treturn ret;\n }\n \n-int refs_verify_refnames_available(struct ref_store *refs,\n-\t\t\t\t const struct string_list *refnames,\n-\t\t\t\t const struct string_list *extras,\n-\t\t\t\t const struct string_list *skip,\n-\t\t\t\t unsigned int initial_transaction,\n-\t\t\t\t struct strbuf *err)\n+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,\n+\t\t\t\t\t const struct string_list *refnames,\n+\t\t\t\t\t const struct string_list *extras,\n+\t\t\t\t\t const struct string_list *skip,\n+\t\t\t\t\t unsigned int initial_transaction,\n+\t\t\t\t\t struct strbuf *err)\n {\n \tstruct strbuf dirname = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct string_list_item *item;\n \tstruct ref_iterator *iter = NULL;\n \tstruct strset dirnames;\n-\tint ret = -1;\n+\tint ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \n \t/*\n \t * For the sake of comments in this function, suppose that\n@@ -2625,12 +2625,13 @@ int refs_verify_refnames_available(struct ref_store *refs,\n \treturn ret;\n }\n \n-int refs_verify_refname_available(struct ref_store *refs,\n-\t\t\t\t const char *refname,\n-\t\t\t\t const struct string_list *extras,\n-\t\t\t\t const struct string_list *skip,\n-\t\t\t\t unsigned int initial_transaction,\n-\t\t\t\t struct strbuf *err)\n+enum ref_transaction_error refs_verify_refname_available(\n+\tstruct ref_store *refs,\n+\tconst char *refname,\n+\tconst struct string_list *extras,\n+\tconst struct string_list *skip,\n+\tunsigned int initial_transaction,\n+\tstruct strbuf *err)\n {\n \tstruct string_list_item item = { .string = (char *) refname };\n \tstruct string_list refnames = {\n@@ -2818,8 +2819,9 @@ int ref_update_has_null_new_value(struct ref_update *update)\n \treturn !update->new_target && is_null_oid(&update->new_oid);\n }\n \n-int ref_update_check_old_target(const char *referent, struct ref_update *update,\n-\t\t\t\tstruct strbuf *err)\n+enum ref_transaction_error ref_update_check_old_target(const char *referent,\n+\t\t\t\t\t\t struct ref_update *update,\n+\t\t\t\t\t\t struct strbuf *err)\n {\n \tif (!update->old_target)\n \t\tBUG(\"called without old_target set\");\n@@ -2827,17 +2829,18 @@ int ref_update_check_old_target(const char *referent, struct ref_update *update,\n \tif (!strcmp(referent, update->old_target))\n \t\treturn 0;\n \n-\tif (!strcmp(referent, \"\"))\n+\tif (!strcmp(referent, \"\")) {\n \t\tstrbuf_addf(err, \"verifying symref target: '%s': \"\n \t\t\t \"reference is missing but expected %s\",\n \t\t\t ref_update_original_update_refname(update),\n \t\t\t update->old_target);\n-\telse\n-\t\tstrbuf_addf(err, \"verifying symref target: '%s': \"\n-\t\t\t \"is at %s but expected %s\",\n+\t\treturn REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n+\t}\n+\n+\tstrbuf_addf(err, \"verifying symref target: '%s': is at %s but expected %s\",\n \t\t\t ref_update_original_update_refname(update),\n \t\t\t referent, update->old_target);\n-\treturn -1;\n+\treturn REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;\n }\n \n struct migration_data {\ndiff --git a/refs.h b/refs.h\nindex 240e2d8537..f009cdae7d 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -16,6 +16,23 @@ struct worktree;\n enum ref_storage_format ref_storage_format_by_name(const char *name);\n const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format);\n \n+enum ref_transaction_error {\n+\t/* Default error code */\n+\tREF_TRANSACTION_ERROR_GENERIC = -1,\n+\t/* Ref name conflict like A vs A/B */\n+\tREF_TRANSACTION_ERROR_NAME_CONFLICT = -2,\n+\t/* Ref to be created already exists */\n+\tREF_TRANSACTION_ERROR_CREATE_EXISTS = -3,\n+\t/* ref expected but doesn't exist */\n+\tREF_TRANSACTION_ERROR_NONEXISTENT_REF = -4,\n+\t/* Provided old_oid or old_target of reference doesn't match actual */\n+\tREF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE = -5,\n+\t/* Provided new_oid or new_target is invalid */\n+\tREF_TRANSACTION_ERROR_INVALID_NEW_VALUE = -6,\n+\t/* Expected ref to be symref, but is a regular ref */\n+\tREF_TRANSACTION_ERROR_EXPECTED_SYMREF = -7,\n+};\n+\n /*\n * Resolve a reference, recursively following symbolic references.\n *\n@@ -117,24 +134,24 @@ int refs_read_symbolic_ref(struct ref_store *ref_store, const char *refname,\n *\n * extras and skip must be sorted.\n */\n-int refs_verify_refname_available(struct ref_store *refs,\n-\t\t\t\t const char *refname,\n-\t\t\t\t const struct string_list *extras,\n-\t\t\t\t const struct string_list *skip,\n-\t\t\t\t unsigned int initial_transaction,\n-\t\t\t\t struct strbuf *err);\n+enum ref_transaction_error refs_verify_refname_available(struct ref_store *refs,\n+\t\t\t\t\t\t const char *refname,\n+\t\t\t\t\t\t const struct string_list *extras,\n+\t\t\t\t\t\t const struct string_list *skip,\n+\t\t\t\t\t\t unsigned int initial_transaction,\n+\t\t\t\t\t\t struct strbuf *err);\n \n /*\n * Same as `refs_verify_refname_available()`, but checking for a list of\n * refnames instead of only a single item. This is more efficient in the case\n * where one needs to check multiple refnames.\n */\n-int refs_verify_refnames_available(struct ref_store *refs,\n-\t\t\t\t const struct string_list *refnames,\n-\t\t\t\t const struct string_list *extras,\n-\t\t\t\t const struct string_list *skip,\n-\t\t\t\t unsigned int initial_transaction,\n-\t\t\t\t struct strbuf *err);\n+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,\n+\t\t\t\t\t const struct string_list *refnames,\n+\t\t\t\t\t const struct string_list *extras,\n+\t\t\t\t\t const struct string_list *skip,\n+\t\t\t\t\t unsigned int initial_transaction,\n+\t\t\t\t\t struct strbuf *err);\n \n int refs_ref_exists(struct ref_store *refs, const char *refname);\n \n@@ -830,13 +847,6 @@ int ref_transaction_verify(struct ref_transaction *transaction,\n \t\t\t unsigned int flags,\n \t\t\t struct strbuf *err);\n \n-/* Naming conflict (for example, the ref names A and A/B conflict). */\n-#define TRANSACTION_NAME_CONFLICT -1\n-/* When only creation was requested, but the ref already exists. */\n-#define TRANSACTION_CREATE_EXISTS -2\n-/* All other errors. */\n-#define TRANSACTION_GENERIC_ERROR -3\n-\n /*\n * Perform the preparatory stages of committing `transaction`. Acquire\n * any needed locks, check preconditions, etc.; basically, do as much\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex ea023a59fc..4f27f7652c 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -663,7 +663,7 @@ static void unlock_ref(struct ref_lock *lock)\n * broken, lock the reference anyway but clear old_oid.\n *\n * Return 0 on success. On failure, write an error message to err and\n- * return TRANSACTION_NAME_CONFLICT or TRANSACTION_GENERIC_ERROR.\n+ * return REF_TRANSACTION_ERROR_NAME_CONFLICT or REF_TRANSACTION_ERROR_GENERIC.\n *\n * Implementation note: This function is basically\n *\n@@ -676,19 +676,20 @@ static void unlock_ref(struct ref_lock *lock)\n * avoided, namely if we were successfully able to read the ref\n * - Generate informative error messages in the case of failure\n */\n-static int lock_raw_ref(struct files_ref_store *refs,\n-\t\t\tconst char *refname, int mustexist,\n-\t\t\tstruct string_list *refnames_to_check,\n-\t\t\tconst struct string_list *extras,\n-\t\t\tstruct ref_lock **lock_p,\n-\t\t\tstruct strbuf *referent,\n-\t\t\tunsigned int *type,\n-\t\t\tstruct strbuf *err)\n-{\n+static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int mustexist,\n+\t\t\t\t\t struct string_list *refnames_to_check,\n+\t\t\t\t\t const struct string_list *extras,\n+\t\t\t\t\t struct ref_lock **lock_p,\n+\t\t\t\t\t struct strbuf *referent,\n+\t\t\t\t\t unsigned int *type,\n+\t\t\t\t\t struct strbuf *err)\n+{\n+\tenum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;\n \tstruct ref_lock *lock;\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tint attempts_remaining = 3;\n-\tint ret = TRANSACTION_GENERIC_ERROR;\n \tint failure_errno;\n \n \tassert(err);\n@@ -728,13 +729,14 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\tstrbuf_reset(err);\n \t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s'\",\n \t\t\t\t\t refname);\n+\t\t\t\tret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n \t\t\t} else {\n \t\t\t\t/*\n \t\t\t\t * The error message set by\n \t\t\t\t * refs_verify_refname_available() is\n \t\t\t\t * OK.\n \t\t\t\t */\n-\t\t\t\tret = TRANSACTION_NAME_CONFLICT;\n+\t\t\t\tret = REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t\t\t}\n \t\t} else {\n \t\t\t/*\n@@ -788,6 +790,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t/* Garden variety missing reference. */\n \t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s'\",\n \t\t\t\t\t refname);\n+\t\t\t\tret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n \t\t\t\tgoto error_return;\n \t\t\t} else {\n \t\t\t\t/*\n@@ -820,6 +823,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t/* Garden variety missing reference. */\n \t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s'\",\n \t\t\t\t\t refname);\n+\t\t\t\tret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n \t\t\t\tgoto error_return;\n \t\t\t} else if (remove_dir_recursively(&ref_file,\n \t\t\t\t\t\t\t REMOVE_DIR_EMPTY_ONLY)) {\n@@ -830,7 +834,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t\t * The error message set by\n \t\t\t\t\t * verify_refname_available() is OK.\n \t\t\t\t\t */\n-\t\t\t\t\tret = TRANSACTION_NAME_CONFLICT;\n+\t\t\t\t\tret = REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t\t\t\t\tgoto error_return;\n \t\t\t\t} else {\n \t\t\t\t\t/*\n@@ -1517,10 +1521,11 @@ static int rename_tmp_log(struct files_ref_store *refs, const char *newrefname)\n \treturn ret;\n }\n \n-static int write_ref_to_lockfile(struct files_ref_store *refs,\n-\t\t\t\t struct ref_lock *lock,\n-\t\t\t\t const struct object_id *oid,\n-\t\t\t\t int skip_oid_verification, struct strbuf *err);\n+static enum ref_transaction_error write_ref_to_lockfile(struct files_ref_store *refs,\n+\t\t\t\t\t\t\tstruct ref_lock *lock,\n+\t\t\t\t\t\t\tconst struct object_id *oid,\n+\t\t\t\t\t\t\tint skip_oid_verification,\n+\t\t\t\t\t\t\tstruct strbuf *err);\n static int commit_ref_update(struct files_ref_store *refs,\n \t\t\t struct ref_lock *lock,\n \t\t\t const struct object_id *oid, const char *logmsg,\n@@ -1926,10 +1931,11 @@ static int files_log_ref_write(struct files_ref_store *refs,\n * Write oid into the open lockfile, then close the lockfile. On\n * errors, rollback the lockfile, fill in *err and return -1.\n */\n-static int write_ref_to_lockfile(struct files_ref_store *refs,\n-\t\t\t\t struct ref_lock *lock,\n-\t\t\t\t const struct object_id *oid,\n-\t\t\t\t int skip_oid_verification, struct strbuf *err)\n+static enum ref_transaction_error write_ref_to_lockfile(struct files_ref_store *refs,\n+\t\t\t\t\t\t\tstruct ref_lock *lock,\n+\t\t\t\t\t\t\tconst struct object_id *oid,\n+\t\t\t\t\t\t\tint skip_oid_verification,\n+\t\t\t\t\t\t\tstruct strbuf *err)\n {\n \tstatic char term = '\\n';\n \tstruct object *o;\n@@ -1943,7 +1949,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,\n \t\t\t\t\"trying to write ref '%s' with nonexistent object %s\",\n \t\t\t\tlock->ref_name, oid_to_hex(oid));\n \t\t\tunlock_ref(lock);\n-\t\t\treturn -1;\n+\t\t\treturn REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;\n \t\t}\n \t\tif (o->type != OBJ_COMMIT && is_branch(lock->ref_name)) {\n \t\t\tstrbuf_addf(\n@@ -1951,7 +1957,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,\n \t\t\t\t\"trying to write non-commit object %s to branch '%s'\",\n \t\t\t\toid_to_hex(oid), lock->ref_name);\n \t\t\tunlock_ref(lock);\n-\t\t\treturn -1;\n+\t\t\treturn REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;\n \t\t}\n \t}\n \tfd = get_lock_file_fd(&lock->lk);\n@@ -1962,7 +1968,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,\n \t\tstrbuf_addf(err,\n \t\t\t \"couldn't write '%s'\", get_lock_file_path(&lock->lk));\n \t\tunlock_ref(lock);\n-\t\treturn -1;\n+\t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \t}\n \treturn 0;\n }\n@@ -2376,9 +2382,10 @@ static struct ref_iterator *files_reflog_iterator_begin(struct ref_store *ref_st\n * If update is a direct update of head_ref (the reference pointed to\n * by HEAD), then add an extra REF_LOG_ONLY update for HEAD.\n */\n-static int split_head_update(struct ref_update *update,\n-\t\t\t struct ref_transaction *transaction,\n-\t\t\t const char *head_ref, struct strbuf *err)\n+static enum ref_transaction_error split_head_update(struct ref_update *update,\n+\t\t\t\t\t\t struct ref_transaction *transaction,\n+\t\t\t\t\t\t const char *head_ref,\n+\t\t\t\t\t\t struct strbuf *err)\n {\n \tstruct ref_update *new_update;\n \n@@ -2402,7 +2409,7 @@ static int split_head_update(struct ref_update *update,\n \t\t\t \"multiple updates for 'HEAD' (including one \"\n \t\t\t \"via its referent '%s') are not allowed\",\n \t\t\t update->refname);\n-\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t\treturn REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t}\n \n \tnew_update = ref_transaction_add_update(\n@@ -2430,10 +2437,10 @@ static int split_head_update(struct ref_update *update,\n * Note that the new update will itself be subject to splitting when\n * the iteration gets to it.\n */\n-static int split_symref_update(struct ref_update *update,\n-\t\t\t const char *referent,\n-\t\t\t struct ref_transaction *transaction,\n-\t\t\t struct strbuf *err)\n+static enum ref_transaction_error split_symref_update(struct ref_update *update,\n+\t\t\t\t\t\t const char *referent,\n+\t\t\t\t\t\t struct ref_transaction *transaction,\n+\t\t\t\t\t\t struct strbuf *err)\n {\n \tstruct ref_update *new_update;\n \tunsigned int new_flags;\n@@ -2450,7 +2457,7 @@ static int split_symref_update(struct ref_update *update,\n \t\t\t \"multiple updates for '%s' (including one \"\n \t\t\t \"via symref '%s') are not allowed\",\n \t\t\t referent, update->refname);\n-\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t\treturn REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t}\n \n \tnew_flags = update->flags;\n@@ -2491,11 +2498,10 @@ static int split_symref_update(struct ref_update *update,\n * everything is OK, return 0; otherwise, write an error message to\n * err and return -1.\n */\n-static int check_old_oid(struct ref_update *update, struct object_id *oid,\n-\t\t\t struct strbuf *err)\n+static enum ref_transaction_error check_old_oid(struct ref_update *update,\n+\t\t\t\t\t\tstruct object_id *oid,\n+\t\t\t\t\t\tstruct strbuf *err)\n {\n-\tint ret = TRANSACTION_GENERIC_ERROR;\n-\n \tif (!(update->flags & REF_HAVE_OLD) ||\n \t\t oideq(oid, &update->old_oid))\n \t\treturn 0;\n@@ -2504,21 +2510,20 @@ static int check_old_oid(struct ref_update *update, struct object_id *oid,\n \t\tstrbuf_addf(err, \"cannot lock ref '%s': \"\n \t\t\t \"reference already exists\",\n \t\t\t ref_update_original_update_refname(update));\n-\t\tret = TRANSACTION_CREATE_EXISTS;\n-\t}\n-\telse if (is_null_oid(oid))\n+\t\treturn REF_TRANSACTION_ERROR_CREATE_EXISTS;\n+\t} else if (is_null_oid(oid)) {\n \t\tstrbuf_addf(err, \"cannot lock ref '%s': \"\n \t\t\t \"reference is missing but expected %s\",\n \t\t\t ref_update_original_update_refname(update),\n \t\t\t oid_to_hex(&update->old_oid));\n-\telse\n-\t\tstrbuf_addf(err, \"cannot lock ref '%s': \"\n-\t\t\t \"is at %s but expected %s\",\n-\t\t\t ref_update_original_update_refname(update),\n-\t\t\t oid_to_hex(oid),\n-\t\t\t oid_to_hex(&update->old_oid));\n+\t\treturn REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n+\t}\n \n-\treturn ret;\n+\tstrbuf_addf(err, \"cannot lock ref '%s': is at %s but expected %s\",\n+\t\t ref_update_original_update_refname(update), oid_to_hex(oid),\n+\t\t oid_to_hex(&update->old_oid));\n+\n+\treturn REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;\n }\n \n struct files_transaction_backend_data {\n@@ -2540,17 +2545,17 @@ struct files_transaction_backend_data {\n * - If it is an update of head_ref, add a corresponding REF_LOG_ONLY\n * update of HEAD.\n */\n-static int lock_ref_for_update(struct files_ref_store *refs,\n-\t\t\t struct ref_update *update,\n-\t\t\t struct ref_transaction *transaction,\n-\t\t\t const char *head_ref,\n-\t\t\t struct string_list *refnames_to_check,\n-\t\t\t struct strbuf *err)\n+static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *refs,\n+\t\t\t\t\t\t struct ref_update *update,\n+\t\t\t\t\t\t struct ref_transaction *transaction,\n+\t\t\t\t\t\t const char *head_ref,\n+\t\t\t\t\t\t struct string_list *refnames_to_check,\n+\t\t\t\t\t\t struct strbuf *err)\n {\n \tstruct strbuf referent = STRBUF_INIT;\n \tint mustexist = ref_update_expects_existing_old_ref(update);\n \tstruct files_transaction_backend_data *backend_data;\n-\tint ret = 0;\n+\tenum ref_transaction_error ret = 0;\n \tstruct ref_lock *lock;\n \n \tfiles_assert_main_repository(refs, \"lock_ref_for_update\");\n@@ -2602,22 +2607,17 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\t\t\t\tstrbuf_addf(err, \"cannot lock ref '%s': \"\n \t\t\t\t\t\t \"error reading reference\",\n \t\t\t\t\t\t ref_update_original_update_refname(update));\n-\t\t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\t\t\tgoto out;\n \t\t\t\t}\n \t\t\t}\n \n-\t\t\tif (update->old_target) {\n-\t\t\t\tif (ref_update_check_old_target(referent.buf, update, err)) {\n-\t\t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n-\t\t\t\t\tgoto out;\n-\t\t\t\t}\n-\t\t\t} else {\n+\t\t\tif (update->old_target)\n+\t\t\t\tret = ref_update_check_old_target(referent.buf, update, err);\n+\t\t\telse\n \t\t\t\tret = check_old_oid(update, &lock->old_oid, err);\n-\t\t\t\tif (ret) {\n-\t\t\t\t\tgoto out;\n-\t\t\t\t}\n-\t\t\t}\n+\t\t\tif (ret)\n+\t\t\t\tgoto out;\n \t\t} else {\n \t\t\t/*\n \t\t\t * Create a new update for the reference this\n@@ -2644,7 +2644,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\t\t\t\t \"but is a regular ref\"),\n \t\t\t\t ref_update_original_update_refname(update),\n \t\t\t\t update->old_target);\n-\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\tret = REF_TRANSACTION_ERROR_EXPECTED_SYMREF;\n \t\t\tgoto out;\n \t\t} else {\n \t\t\tret = check_old_oid(update, &lock->old_oid, err);\n@@ -2668,14 +2668,14 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \n \tif (update->new_target && !(update->flags & REF_LOG_ONLY)) {\n \t\tif (create_symref_lock(lock, update->new_target, err)) {\n-\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\tgoto out;\n \t\t}\n \n \t\tif (close_ref_gently(lock)) {\n \t\t\tstrbuf_addf(err, \"couldn't close '%s.lock'\",\n \t\t\t\t update->refname);\n-\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\tgoto out;\n \t\t}\n \n@@ -2693,25 +2693,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\t\t * The reference already has the desired\n \t\t\t * value, so we don't need to write it.\n \t\t\t */\n-\t\t} else if (write_ref_to_lockfile(\n-\t\t\t\t refs, lock, &update->new_oid,\n-\t\t\t\t update->flags & REF_SKIP_OID_VERIFICATION,\n-\t\t\t\t err)) {\n-\t\t\tchar *write_err = strbuf_detach(err, NULL);\n-\n-\t\t\t/*\n-\t\t\t * The lock was freed upon failure of\n-\t\t\t * write_ref_to_lockfile():\n-\t\t\t */\n-\t\t\tupdate->backend_data = NULL;\n-\t\t\tstrbuf_addf(err,\n-\t\t\t\t \"cannot update ref '%s': %s\",\n-\t\t\t\t update->refname, write_err);\n-\t\t\tfree(write_err);\n-\t\t\tret = TRANSACTION_GENERIC_ERROR;\n-\t\t\tgoto out;\n \t\t} else {\n-\t\t\tupdate->flags |= REF_NEEDS_COMMIT;\n+\t\t\tret = write_ref_to_lockfile(\n+\t\t\t\trefs, lock, &update->new_oid,\n+\t\t\t\tupdate->flags & REF_SKIP_OID_VERIFICATION,\n+\t\t\t\terr);\n+\t\t\tif (ret) {\n+\t\t\t\tchar *write_err = strbuf_detach(err, NULL);\n+\n+\t\t\t\t/*\n+\t\t\t\t * The lock was freed upon failure of\n+\t\t\t\t * write_ref_to_lockfile():\n+\t\t\t\t */\n+\t\t\t\tupdate->backend_data = NULL;\n+\t\t\t\tstrbuf_addf(err,\n+\t\t\t\t\t \"cannot update ref '%s': %s\",\n+\t\t\t\t\t update->refname, write_err);\n+\t\t\t\tfree(write_err);\n+\t\t\t\tgoto out;\n+\t\t\t} else {\n+\t\t\t\tupdate->flags |= REF_NEEDS_COMMIT;\n+\t\t\t}\n \t\t}\n \t}\n \tif (!(update->flags & REF_NEEDS_COMMIT)) {\n@@ -2723,7 +2725,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\tif (close_ref_gently(lock)) {\n \t\t\tstrbuf_addf(err, \"couldn't close '%s.lock'\",\n \t\t\t\t update->refname);\n-\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\tgoto out;\n \t\t}\n \t}\n@@ -2865,7 +2867,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\t\t\trefs->packed_ref_store,\n \t\t\t\t\t\ttransaction->flags, err);\n \t\t\t\tif (!packed_transaction) {\n-\t\t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\t\t\tgoto cleanup;\n \t\t\t\t}\n \n@@ -2897,13 +2899,13 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t */\n \tif (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,\n \t\t\t\t\t &transaction->refnames, NULL, 0, err)) {\n-\t\tret = TRANSACTION_NAME_CONFLICT;\n+\t\tret = REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t\tgoto cleanup;\n \t}\n \n \tif (packed_transaction) {\n \t\tif (packed_refs_lock(refs->packed_ref_store, 0, err)) {\n-\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\tgoto cleanup;\n \t\t}\n \t\tbackend_data->packed_refs_locked = 1;\n@@ -2934,7 +2936,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t\t\t */\n \t\t\tbackend_data->packed_transaction = NULL;\n \t\t\tif (ref_transaction_abort(packed_transaction, err)) {\n-\t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\t\tgoto cleanup;\n \t\t\t}\n \t\t}\n@@ -3035,7 +3037,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \tpacked_transaction = ref_store_transaction_begin(refs->packed_ref_store,\n \t\t\t\t\t\t\t transaction->flags, err);\n \tif (!packed_transaction) {\n-\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\tgoto cleanup;\n \t}\n \n@@ -3058,7 +3060,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \t\t\tif (!loose_transaction) {\n \t\t\t\tloose_transaction = ref_store_transaction_begin(&refs->base, 0, err);\n \t\t\t\tif (!loose_transaction) {\n-\t\t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\t\t\tgoto cleanup;\n \t\t\t\t}\n \t\t\t}\n@@ -3083,19 +3085,19 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \t}\n \n \tif (packed_refs_lock(refs->packed_ref_store, 0, err)) {\n-\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\tgoto cleanup;\n \t}\n \n \tif (refs_verify_refnames_available(&refs->base, &refnames_to_check,\n \t\t\t\t\t &affected_refnames, NULL, 1, err)) {\n \t\tpacked_refs_unlock(refs->packed_ref_store);\n-\t\tret = TRANSACTION_NAME_CONFLICT;\n+\t\tret = REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t\tgoto cleanup;\n \t}\n \n \tif (ref_transaction_commit(packed_transaction, err)) {\n-\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\tgoto cleanup;\n \t}\n \tpacked_refs_unlock(refs->packed_ref_store);\n@@ -3103,7 +3105,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \tif (loose_transaction) {\n \t\tif (ref_transaction_prepare(loose_transaction, err) ||\n \t\t ref_transaction_commit(loose_transaction, err)) {\n-\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\tgoto cleanup;\n \t\t}\n \t}\n@@ -3152,7 +3154,7 @@ static int files_transaction_finish(struct ref_store *ref_store,\n \t\tif (update->flags & REF_NEEDS_COMMIT ||\n \t\t update->flags & REF_LOG_ONLY) {\n \t\t\tif (parse_and_write_reflog(refs, update, lock, err)) {\n-\t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\t\tgoto cleanup;\n \t\t\t}\n \t\t}\n@@ -3171,7 +3173,7 @@ static int files_transaction_finish(struct ref_store *ref_store,\n \t\t\t\tstrbuf_addf(err, \"couldn't set '%s'\", lock->ref_name);\n \t\t\t\tunlock_ref(lock);\n \t\t\t\tupdate->backend_data = NULL;\n-\t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\t\tgoto cleanup;\n \t\t\t}\n \t\t}\n@@ -3227,7 +3229,7 @@ static int files_transaction_finish(struct ref_store *ref_store,\n \t\t\t\tstrbuf_reset(&sb);\n \t\t\t\tfiles_ref_path(refs, &sb, lock->ref_name);\n \t\t\t\tif (unlink_or_msg(sb.buf, err)) {\n-\t\t\t\t\tret = TRANSACTION_GENERIC_ERROR;\n+\t\t\t\t\tret = REF_TRANSACTION_ERROR_GENERIC;\n \t\t\t\t\tgoto cleanup;\n \t\t\t\t}\n \t\t\t}\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 19220d2e99..d90bd815a3 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1326,10 +1326,11 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,\n * The packfile must be locked before calling this function and will\n * remain locked when it is done.\n */\n-static int write_with_updates(struct packed_ref_store *refs,\n-\t\t\t struct string_list *updates,\n-\t\t\t struct strbuf *err)\n+static enum ref_transaction_error write_with_updates(struct packed_ref_store *refs,\n+\t\t\t\t\t\t struct string_list *updates,\n+\t\t\t\t\t\t struct strbuf *err)\n {\n+\tenum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;\n \tstruct ref_iterator *iter = NULL;\n \tsize_t i;\n \tint ok;\n@@ -1353,7 +1354,7 @@ static int write_with_updates(struct packed_ref_store *refs,\n \t\tstrbuf_addf(err, \"unable to create file %s: %s\",\n \t\t\t sb.buf, strerror(errno));\n \t\tstrbuf_release(&sb);\n-\t\treturn -1;\n+\t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \t}\n \tstrbuf_release(&sb);\n \n@@ -1409,6 +1410,7 @@ static int write_with_updates(struct packed_ref_store *refs,\n \t\t\t\t\tstrbuf_addf(err, \"cannot update ref '%s': \"\n \t\t\t\t\t\t \"reference already exists\",\n \t\t\t\t\t\t update->refname);\n+\t\t\t\t\tret = REF_TRANSACTION_ERROR_CREATE_EXISTS;\n \t\t\t\t\tgoto error;\n \t\t\t\t} else if (!oideq(&update->old_oid, iter->oid)) {\n \t\t\t\t\tstrbuf_addf(err, \"cannot update ref '%s': \"\n@@ -1416,6 +1418,7 @@ static int write_with_updates(struct packed_ref_store *refs,\n \t\t\t\t\t\t update->refname,\n \t\t\t\t\t\t oid_to_hex(iter->oid),\n \t\t\t\t\t\t oid_to_hex(&update->old_oid));\n+\t\t\t\t\tret = REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;\n \t\t\t\t\tgoto error;\n \t\t\t\t}\n \t\t\t}\n@@ -1452,6 +1455,7 @@ static int write_with_updates(struct packed_ref_store *refs,\n \t\t\t\t\t \"reference is missing but expected %s\",\n \t\t\t\t\t update->refname,\n \t\t\t\t\t oid_to_hex(&update->old_oid));\n+\t\t\t\tret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n \t\t\t\tgoto error;\n \t\t\t}\n \t\t}\n@@ -1509,7 +1513,7 @@ static int write_with_updates(struct packed_ref_store *refs,\n \t\t\t strerror(errno));\n \t\tstrbuf_release(&sb);\n \t\tdelete_tempfile(&refs->tempfile);\n-\t\treturn -1;\n+\t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \t}\n \n \treturn 0;\n@@ -1521,7 +1525,7 @@ static int write_with_updates(struct packed_ref_store *refs,\n error:\n \tref_iterator_free(iter);\n \tdelete_tempfile(&refs->tempfile);\n-\treturn -1;\n+\treturn ret;\n }\n \n int is_packed_transaction_needed(struct ref_store *ref_store,\n@@ -1654,7 +1658,7 @@ static int packed_transaction_prepare(struct ref_store *ref_store,\n \t\t\tREF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,\n \t\t\t\"ref_transaction_prepare\");\n \tstruct packed_transaction_backend_data *data;\n-\tint ret = TRANSACTION_GENERIC_ERROR;\n+\tenum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;\n \n \t/*\n \t * Note that we *don't* skip transactions with zero updates,\n@@ -1675,7 +1679,8 @@ static int packed_transaction_prepare(struct ref_store *ref_store,\n \t\tdata->own_lock = 1;\n \t}\n \n-\tif (write_with_updates(refs, &transaction->refnames, err))\n+\tret = write_with_updates(refs, &transaction->refnames, err);\n+\tif (ret)\n \t\tgoto failure;\n \n \ttransaction->state = REF_TRANSACTION_PREPARED;\n@@ -1707,7 +1712,7 @@ static int packed_transaction_finish(struct ref_store *ref_store,\n \t\t\tref_store,\n \t\t\tREF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,\n \t\t\t\"ref_transaction_finish\");\n-\tint ret = TRANSACTION_GENERIC_ERROR;\n+\tint ret = REF_TRANSACTION_ERROR_GENERIC;\n \tchar *packed_refs_path;\n \n \tclear_snapshot(refs);\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 6d3770d0cc..3f1d19abd9 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -770,8 +770,9 @@ int ref_update_has_null_new_value(struct ref_update *update);\n * If everything is OK, return 0; otherwise, write an error message to\n * err and return -1.\n */\n-int ref_update_check_old_target(const char *referent, struct ref_update *update,\n-\t\t\t\tstruct strbuf *err);\n+enum ref_transaction_error ref_update_check_old_target(const char *referent,\n+\t\t\t\t\t\t struct ref_update *update,\n+\t\t\t\t\t\t struct strbuf *err);\n \n /*\n * Check if the ref must exist, this means that the old_oid or\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex 786df11a03..bd6b042103 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -1069,20 +1069,20 @@ static int queue_transaction_update(struct reftable_ref_store *refs,\n \treturn 0;\n }\n \n-static int prepare_single_update(struct reftable_ref_store *refs,\n-\t\t\t\t struct reftable_transaction_data *tx_data,\n-\t\t\t\t struct ref_transaction *transaction,\n-\t\t\t\t struct reftable_backend *be,\n-\t\t\t\t struct ref_update *u,\n-\t\t\t\t struct string_list *refnames_to_check,\n-\t\t\t\t unsigned int head_type,\n-\t\t\t\t struct strbuf *head_referent,\n-\t\t\t\t struct strbuf *referent,\n-\t\t\t\t struct strbuf *err)\n+static enum ref_transaction_error prepare_single_update(struct reftable_ref_store *refs,\n+\t\t\t\t\t\t\tstruct reftable_transaction_data *tx_data,\n+\t\t\t\t\t\t\tstruct ref_transaction *transaction,\n+\t\t\t\t\t\t\tstruct reftable_backend *be,\n+\t\t\t\t\t\t\tstruct ref_update *u,\n+\t\t\t\t\t\t\tstruct string_list *refnames_to_check,\n+\t\t\t\t\t\t\tunsigned int head_type,\n+\t\t\t\t\t\t\tstruct strbuf *head_referent,\n+\t\t\t\t\t\t\tstruct strbuf *referent,\n+\t\t\t\t\t\t\tstruct strbuf *err)\n {\n+\tenum ref_transaction_error ret = 0;\n \tstruct object_id current_oid = {0};\n \tconst char *rewritten_ref;\n-\tint ret = 0;\n \n \t/*\n \t * There is no need to reload the respective backends here as\n@@ -1093,7 +1093,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \t */\n \tret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);\n \tif (ret)\n-\t\treturn ret;\n+\t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \n \t/* Verify that the new object ID is valid. */\n \tif ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&\n@@ -1104,13 +1104,13 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \t\t\tstrbuf_addf(err,\n \t\t\t\t _(\"trying to write ref '%s' with nonexistent object %s\"),\n \t\t\t\t u->refname, oid_to_hex(&u->new_oid));\n-\t\t\treturn -1;\n+\t\t\treturn REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;\n \t\t}\n \n \t\tif (o->type != OBJ_COMMIT && is_branch(u->refname)) {\n \t\t\tstrbuf_addf(err, _(\"trying to write non-commit object %s to branch '%s'\"),\n \t\t\t\t oid_to_hex(&u->new_oid), u->refname);\n-\t\t\treturn -1;\n+\t\t\treturn REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;\n \t\t}\n \t}\n \n@@ -1134,7 +1134,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \t\t\t\t _(\"multiple updates for 'HEAD' (including one \"\n \t\t\t\t \"via its referent '%s') are not allowed\"),\n \t\t\t\t u->refname);\n-\t\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t\t\treturn REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t\t}\n \n \t\tref_transaction_add_update(\n@@ -1147,7 +1147,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \tret = reftable_backend_read_ref(be, rewritten_ref,\n \t\t\t\t\t¤t_oid, referent, &u->type);\n \tif (ret < 0)\n-\t\treturn ret;\n+\t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \tif (ret > 0 && !ref_update_expects_existing_old_ref(u)) {\n \t\t/*\n \t\t * The reference does not exist, and we either have no\n@@ -1168,7 +1168,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \t\t\tret = queue_transaction_update(refs, tx_data, u,\n \t\t\t\t\t\t ¤t_oid, err);\n \t\t\tif (ret)\n-\t\t\t\treturn ret;\n+\t\t\t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \t\t}\n \n \t\treturn 0;\n@@ -1180,7 +1180,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \n \t\t\t\t \"unable to resolve reference '%s'\"),\n \t\t\t ref_update_original_update_refname(u), u->refname);\n-\t\treturn -1;\n+\t\treturn REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n \t}\n \n \tif (u->type & REF_ISSYMREF) {\n@@ -1196,7 +1196,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \t\t\tif (u->flags & REF_HAVE_OLD && !resolved) {\n \t\t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n \t\t\t\t\t\t \"error reading reference\"), u->refname);\n-\t\t\t\treturn -1;\n+\t\t\t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \t\t\t}\n \t\t} else {\n \t\t\tstruct ref_update *new_update;\n@@ -1211,7 +1211,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \t\t\t\t\t _(\"multiple updates for '%s' (including one \"\n \t\t\t\t\t \"via symref '%s') are not allowed\"),\n \t\t\t\t\t referent->buf, u->refname);\n-\t\t\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t\t\t\treturn REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t\t\t}\n \n \t\t\t/*\n@@ -1255,31 +1255,32 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \t\t\t\t\t \"but is a regular ref\"),\n \t\t\t\t ref_update_original_update_refname(u),\n \t\t\t\t u->old_target);\n-\t\t\treturn -1;\n+\t\t\treturn REF_TRANSACTION_ERROR_EXPECTED_SYMREF;\n \t\t}\n \n-\t\tif (ref_update_check_old_target(referent->buf, u, err)) {\n-\t\t\treturn -1;\n-\t\t}\n+\t\tret = ref_update_check_old_target(referent->buf, u, err);\n+\t\tif (ret)\n+\t\t\treturn ret;\n \t} else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {\n \t\tif (is_null_oid(&u->old_oid)) {\n \t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n \t\t\t\t\t \"reference already exists\"),\n \t\t\t\t ref_update_original_update_refname(u));\n-\t\t\treturn TRANSACTION_CREATE_EXISTS;\n-\t\t}\n-\t\telse if (is_null_oid(¤t_oid))\n+\t\t\treturn REF_TRANSACTION_ERROR_CREATE_EXISTS;\n+\t\t} else if (is_null_oid(¤t_oid)) {\n \t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n \t\t\t\t\t \"reference is missing but expected %s\"),\n \t\t\t\t ref_update_original_update_refname(u),\n \t\t\t\t oid_to_hex(&u->old_oid));\n-\t\telse\n+\t\t\treturn REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n+\t\t} else {\n \t\t\tstrbuf_addf(err, _(\"cannot lock ref '%s': \"\n \t\t\t\t\t \"is at %s but expected %s\"),\n \t\t\t\t ref_update_original_update_refname(u),\n \t\t\t\t oid_to_hex(¤t_oid),\n \t\t\t\t oid_to_hex(&u->old_oid));\n-\t\treturn TRANSACTION_NAME_CONFLICT;\n+\t\t\treturn REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;\n+\t\t}\n \t}\n \n \t/*\n@@ -1296,8 +1297,8 @@ static int prepare_single_update(struct reftable_ref_store *refs,\n \tif ((u->type & REF_ISSYMREF) ||\n \t (u->flags & REF_LOG_ONLY) ||\n \t (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid)))\n-\t\treturn queue_transaction_update(refs, tx_data, u,\n-\t\t\t\t\t ¤t_oid, err);\n+\t\tif (queue_transaction_update(refs, tx_data, u, ¤t_oid, err))\n+\t\t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \n \treturn 0;\n }\n@@ -1385,7 +1386,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \ttransaction->state = REF_TRANSACTION_PREPARED;\n \n done:\n-\tassert(ret != REFTABLE_API_ERROR);\n \tif (ret < 0) {\n \t\tfree_transaction_data(tx_data);\n \t\ttransaction->state = REF_TRANSACTION_CLOSED;\n-- \n2.48.1\n\n"},{"id":"515840","messageId":"20250408085120.614893-7-karthik.188@gmail.com","threadId":"62915","inReplyTo":"20250408085120.614893-1-karthik.188@gmail.com","subject":"[PATCH v6 6/8] refs: implement batch reference update support","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-08T08:51:10Z","receivedAt":"2025-04-08T08:51:32Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Git supports making reference updates with or without transactions.\nUpdates with transactions are generally better optimized. But\ntransactions are all or nothing. This means, if a user wants to batch\nupdates to take advantage of the optimizations without the hard\nrequirement that all updates must succeed, there is no way currently to\ndo so. Particularly with the reftable backend where batching multiple\nreference updates is more efficient than performing them sequentially.\n\nIntroduce batched update support with a new flag,\n'REF_TRANSACTION_ALLOW_FAILURE'. Batched updates while different from\ntransactions, use the transaction infrastructure under the hood. When\nenabled, this flag allows individual reference updates that would\ntypically cause the entire transaction to fail due to non-system-related\nerrors to be marked as rejected while permitting other updates to\nproceed. System errors referred by 'REF_TRANSACTION_ERROR_GENERIC'\ncontinue to result in the entire transaction failing. This approach\nenhances flexibility while preserving transactional integrity where\nnecessary.\n\nThe implementation introduces several key components:\n\n - Add 'rejection_err' field to struct `ref_update` to track failed\n updates with failure reason.\n\n - Add a new struct `ref_transaction_rejections` and a field within\n `ref_transaction` to this struct to allow quick iteration over\n rejected updates.\n\n - Modify reference backends (files, packed, reftable) to handle\n partial transactions by using `ref_transaction_set_rejected()`\n instead of failing the entire transaction when\n `REF_TRANSACTION_ALLOW_FAILURE` is set.\n\n - Add `ref_transaction_for_each_rejected_update()` to let callers\n examine which updates were rejected and why.\n\nThis foundational change enables batched update support throughout the\nreference subsystem. A following commit will expose this capability to\nusers by adding a `--batch-updates` flag to 'git-update-ref(1)',\nproviding both a user-facing feature and a testable implementation.\n\nSigned-off-by: Karthik Nayak \n---\n refs.c | 61 +++++++++++++++++++++++++++++++++++++++++\n refs.h | 22 +++++++++++++++\n refs/files-backend.c | 12 +++++++-\n refs/packed-backend.c | 27 ++++++++++++++++--\n refs/refs-internal.h | 26 ++++++++++++++++++\n refs/reftable-backend.c | 12 +++++++-\n 6 files changed, 156 insertions(+), 4 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 3d0b53d56e..b34ec198f5 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1176,6 +1176,10 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,\n \ttr->ref_store = refs;\n \ttr->flags = flags;\n \tstring_list_init_dup(&tr->refnames);\n+\n+\tif (flags & REF_TRANSACTION_ALLOW_FAILURE)\n+\t\tCALLOC_ARRAY(tr->rejections, 1);\n+\n \treturn tr;\n }\n \n@@ -1206,11 +1210,45 @@ void ref_transaction_free(struct ref_transaction *transaction)\n \t\tfree((char *)transaction->updates[i]->old_target);\n \t\tfree(transaction->updates[i]);\n \t}\n+\n+\tif (transaction->rejections)\n+\t\tfree(transaction->rejections->update_indices);\n+\tfree(transaction->rejections);\n+\n \tstring_list_clear(&transaction->refnames, 0);\n \tfree(transaction->updates);\n \tfree(transaction);\n }\n \n+int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction,\n+\t\t\t\t size_t update_idx,\n+\t\t\t\t enum ref_transaction_error err)\n+{\n+\tif (update_idx >= transaction->nr)\n+\t\tBUG(\"trying to set rejection on invalid update index\");\n+\n+\tif (!(transaction->flags & REF_TRANSACTION_ALLOW_FAILURE))\n+\t\treturn 0;\n+\n+\tif (!transaction->rejections)\n+\t\tBUG(\"transaction not inititalized with failure support\");\n+\n+\t/*\n+\t * Don't accept generic errors, since these errors are not user\n+\t * input related.\n+\t */\n+\tif (err == REF_TRANSACTION_ERROR_GENERIC)\n+\t\treturn 0;\n+\n+\ttransaction->updates[update_idx]->rejection_err = err;\n+\tALLOC_GROW(transaction->rejections->update_indices,\n+\t\t transaction->rejections->nr + 1,\n+\t\t transaction->rejections->alloc);\n+\ttransaction->rejections->update_indices[transaction->rejections->nr++] = update_idx;\n+\n+\treturn 1;\n+}\n+\n struct ref_update *ref_transaction_add_update(\n \t\tstruct ref_transaction *transaction,\n \t\tconst char *refname, unsigned int flags,\n@@ -1236,6 +1274,7 @@ struct ref_update *ref_transaction_add_update(\n \ttransaction->updates[transaction->nr++] = update;\n \n \tupdate->flags = flags;\n+\tupdate->rejection_err = 0;\n \n \tupdate->new_target = xstrdup_or_null(new_target);\n \tupdate->old_target = xstrdup_or_null(old_target);\n@@ -2728,6 +2767,28 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,\n \t}\n }\n \n+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,\n+\t\t\t\t\t ref_transaction_for_each_rejected_update_fn cb,\n+\t\t\t\t\t void *cb_data)\n+{\n+\tif (!transaction->rejections)\n+\t\treturn;\n+\n+\tfor (size_t i = 0; i < transaction->rejections->nr; i++) {\n+\t\tsize_t update_index = transaction->rejections->update_indices[i];\n+\t\tstruct ref_update *update = transaction->updates[update_index];\n+\n+\t\tif (!update->rejection_err)\n+\t\t\tcontinue;\n+\n+\t\tcb(update->refname,\n+\t\t (update->flags & REF_HAVE_OLD) ? &update->old_oid : NULL,\n+\t\t (update->flags & REF_HAVE_NEW) ? &update->new_oid : NULL,\n+\t\t update->old_target, update->new_target,\n+\t\t update->rejection_err, cb_data);\n+\t}\n+}\n+\n int refs_delete_refs(struct ref_store *refs, const char *logmsg,\n \t\t struct string_list *refnames, unsigned int flags)\n {\ndiff --git a/refs.h b/refs.h\nindex f009cdae7d..c48c800478 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -667,6 +667,13 @@ enum ref_transaction_flag {\n \t * either be absent or null_oid.\n \t */\n \tREF_TRANSACTION_FLAG_INITIAL = (1 << 0),\n+\n+\t/*\n+\t * The transaction mechanism by default fails all updates if any conflict\n+\t * is detected. This flag allows transactions to partially apply updates\n+\t * while rejecting updates which do not match the expected state.\n+\t */\n+\tREF_TRANSACTION_ALLOW_FAILURE = (1 << 1),\n };\n \n /*\n@@ -897,6 +904,21 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,\n \t\t\t\t\t ref_transaction_for_each_queued_update_fn cb,\n \t\t\t\t\t void *cb_data);\n \n+/*\n+ * Execute the given callback function for each of the reference updates which\n+ * have been rejected in the given transaction.\n+ */\n+typedef void ref_transaction_for_each_rejected_update_fn(const char *refname,\n+\t\t\t\t\t\t\t const struct object_id *old_oid,\n+\t\t\t\t\t\t\t const struct object_id *new_oid,\n+\t\t\t\t\t\t\t const char *old_target,\n+\t\t\t\t\t\t\t const char *new_target,\n+\t\t\t\t\t\t\t enum ref_transaction_error err,\n+\t\t\t\t\t\t\t void *cb_data);\n+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,\n+\t\t\t\t\t ref_transaction_for_each_rejected_update_fn cb,\n+\t\t\t\t\t void *cb_data);\n+\n /*\n * Free `*transaction` and all associated data.\n */\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 4f27f7652c..256c69b942 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -2852,8 +2852,15 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t\tret = lock_ref_for_update(refs, update, transaction,\n \t\t\t\t\t head_ref, &refnames_to_check,\n \t\t\t\t\t err);\n-\t\tif (ret)\n+\t\tif (ret) {\n+\t\t\tif (ref_transaction_maybe_set_rejected(transaction, i, ret)) {\n+\t\t\t\tstrbuf_reset(err);\n+\t\t\t\tret = 0;\n+\n+\t\t\t\tcontinue;\n+\t\t\t}\n \t\t\tgoto cleanup;\n+\t\t}\n \n \t\tif (update->flags & REF_DELETING &&\n \t\t !(update->flags & REF_LOG_ONLY) &&\n@@ -3151,6 +3158,9 @@ static int files_transaction_finish(struct ref_store *ref_store,\n \t\tstruct ref_update *update = transaction->updates[i];\n \t\tstruct ref_lock *lock = update->backend_data;\n \n+\t\tif (update->rejection_err)\n+\t\t\tcontinue;\n+\n \t\tif (update->flags & REF_NEEDS_COMMIT ||\n \t\t update->flags & REF_LOG_ONLY) {\n \t\t\tif (parse_and_write_reflog(refs, update, lock, err)) {\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex d90bd815a3..debca86a2b 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1327,10 +1327,11 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,\n * remain locked when it is done.\n */\n static enum ref_transaction_error write_with_updates(struct packed_ref_store *refs,\n-\t\t\t\t\t\t struct string_list *updates,\n+\t\t\t\t\t\t struct ref_transaction *transaction,\n \t\t\t\t\t\t struct strbuf *err)\n {\n \tenum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;\n+\tstruct string_list *updates = &transaction->refnames;\n \tstruct ref_iterator *iter = NULL;\n \tsize_t i;\n \tint ok;\n@@ -1411,6 +1412,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re\n \t\t\t\t\t\t \"reference already exists\",\n \t\t\t\t\t\t update->refname);\n \t\t\t\t\tret = REF_TRANSACTION_ERROR_CREATE_EXISTS;\n+\n+\t\t\t\t\tif (ref_transaction_maybe_set_rejected(transaction, i, ret)) {\n+\t\t\t\t\t\tstrbuf_reset(err);\n+\t\t\t\t\t\tret = 0;\n+\t\t\t\t\t\tcontinue;\n+\t\t\t\t\t}\n+\n \t\t\t\t\tgoto error;\n \t\t\t\t} else if (!oideq(&update->old_oid, iter->oid)) {\n \t\t\t\t\tstrbuf_addf(err, \"cannot update ref '%s': \"\n@@ -1419,6 +1427,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re\n \t\t\t\t\t\t oid_to_hex(iter->oid),\n \t\t\t\t\t\t oid_to_hex(&update->old_oid));\n \t\t\t\t\tret = REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;\n+\n+\t\t\t\t\tif (ref_transaction_maybe_set_rejected(transaction, i, ret)) {\n+\t\t\t\t\t\tstrbuf_reset(err);\n+\t\t\t\t\t\tret = 0;\n+\t\t\t\t\t\tcontinue;\n+\t\t\t\t\t}\n+\n \t\t\t\t\tgoto error;\n \t\t\t\t}\n \t\t\t}\n@@ -1456,6 +1471,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re\n \t\t\t\t\t update->refname,\n \t\t\t\t\t oid_to_hex(&update->old_oid));\n \t\t\t\tret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;\n+\n+\t\t\t\tif (ref_transaction_maybe_set_rejected(transaction, i, ret)) {\n+\t\t\t\t\tstrbuf_reset(err);\n+\t\t\t\t\tret = 0;\n+\t\t\t\t\tcontinue;\n+\t\t\t\t}\n+\n \t\t\t\tgoto error;\n \t\t\t}\n \t\t}\n@@ -1521,6 +1543,7 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re\n write_error:\n \tstrbuf_addf(err, \"error writing to %s: %s\",\n \t\t get_tempfile_path(refs->tempfile), strerror(errno));\n+\tret = REF_TRANSACTION_ERROR_GENERIC;\n \n error:\n \tref_iterator_free(iter);\n@@ -1679,7 +1702,7 @@ static int packed_transaction_prepare(struct ref_store *ref_store,\n \t\tdata->own_lock = 1;\n \t}\n \n-\tret = write_with_updates(refs, &transaction->refnames, err);\n+\tret = write_with_updates(refs, transaction, err);\n \tif (ret)\n \t\tgoto failure;\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 3f1d19abd9..73a5379b73 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -123,6 +123,12 @@ struct ref_update {\n \t */\n \tuint64_t index;\n \n+\t/*\n+\t * Used in batched reference updates to mark if a given update\n+\t * was rejected.\n+\t */\n+\tenum ref_transaction_error rejection_err;\n+\n \t/*\n \t * If this ref_update was split off of a symref update via\n \t * split_symref_update(), then this member points at that\n@@ -142,6 +148,13 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t struct object_id *oid, struct strbuf *referent,\n \t\t unsigned int *type, int *failure_errno);\n \n+/*\n+ * Mark a given update as rejected with a given reason.\n+ */\n+int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction,\n+\t\t\t\t size_t update_idx,\n+\t\t\t\t enum ref_transaction_error err);\n+\n /*\n * Add a ref_update with the specified properties to transaction, and\n * return a pointer to the new object. This function does not verify\n@@ -183,6 +196,18 @@ enum ref_transaction_state {\n \tREF_TRANSACTION_CLOSED = 2\n };\n \n+/*\n+ * Data structure to hold indices of updates which were rejected, for batched\n+ * reference updates. While the updates themselves hold the rejection error,\n+ * this structure allows a transaction to iterate only over the rejected\n+ * updates.\n+ */\n+struct ref_transaction_rejections {\n+\tsize_t *update_indices;\n+\tsize_t alloc;\n+\tsize_t nr;\n+};\n+\n /*\n * Data structure for holding a reference transaction, which can\n * consist of checks and updates to multiple references, carried out\n@@ -195,6 +220,7 @@ struct ref_transaction {\n \tsize_t alloc;\n \tsize_t nr;\n \tenum ref_transaction_state state;\n+\tstruct ref_transaction_rejections *rejections;\n \tvoid *backend_data;\n \tunsigned int flags;\n \tuint64_t max_index;\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex bd6b042103..5db4a108b9 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -1371,8 +1371,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t\t\t\t\t transaction->updates[i],\n \t\t\t\t\t &refnames_to_check, head_type,\n \t\t\t\t\t &head_referent, &referent, err);\n-\t\tif (ret)\n+\t\tif (ret) {\n+\t\t\tif (ref_transaction_maybe_set_rejected(transaction, i, ret)) {\n+\t\t\t\tstrbuf_reset(err);\n+\t\t\t\tret = 0;\n+\n+\t\t\t\tcontinue;\n+\t\t\t}\n \t\t\tgoto done;\n+\t\t}\n \t}\n \n \tret = refs_verify_refnames_available(ref_store, &refnames_to_check,\n@@ -1454,6 +1461,9 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data\n \t\tstruct reftable_transaction_update *tx_update = &arg->updates[i];\n \t\tstruct ref_update *u = tx_update->update;\n \n+\t\tif (u->rejection_err)\n+\t\t\tcontinue;\n+\n \t\t/*\n \t\t * Write a reflog entry when updating a ref to point to\n \t\t * something new in either of the following cases:\n-- \n2.48.1\n\n"},{"id":"515842","messageId":"20250408085120.614893-8-karthik.188@gmail.com","threadId":"62915","inReplyTo":"20250408085120.614893-1-karthik.188@gmail.com","subject":"[PATCH v6 7/8] refs: support rejection in batch updates during F/D checks","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-08T08:51:11Z","receivedAt":"2025-04-08T08:51:32Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"The `refs_verify_refnames_available()` is used to batch check refnames\nfor F/D conflicts. While this is the more performant alternative than\nits individual version, it does not provide rejection capabilities on a\nsingle update level. For batched updates, this would mean a rejection of\nthe entire transaction whenever one reference has a F/D conflict.\n\nModify the function to call `ref_transaction_maybe_set_rejected()` to\ncheck if a single update can be rejected. Since this function is only\ninternally used within 'refs/' and we want to pass in a `struct\nref_transaction *` as a variable. We also move and mark\n`refs_verify_refnames_available()` to 'refs-internal.h' to be an\ninternal function.\n\nSigned-off-by: Karthik Nayak \n---\n refs.c | 37 ++++++++++++++++++++++++++++++++++---\n refs.h | 12 ------------\n refs/files-backend.c | 27 ++++++++++++++++++---------\n refs/refs-internal.h | 16 ++++++++++++++++\n refs/reftable-backend.c | 11 ++++++++---\n 5 files changed, 76 insertions(+), 27 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex b34ec198f5..41d6247e70 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2540,6 +2540,7 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs\n \t\t\t\t\t const struct string_list *refnames,\n \t\t\t\t\t const struct string_list *extras,\n \t\t\t\t\t const struct string_list *skip,\n+\t\t\t\t\t struct ref_transaction *transaction,\n \t\t\t\t\t unsigned int initial_transaction,\n \t\t\t\t\t struct strbuf *err)\n {\n@@ -2547,6 +2548,7 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct string_list_item *item;\n \tstruct ref_iterator *iter = NULL;\n+\tstruct strset conflicting_dirnames;\n \tstruct strset dirnames;\n \tint ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \n@@ -2557,9 +2559,11 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs\n \n \tassert(err);\n \n+\tstrset_init(&conflicting_dirnames);\n \tstrset_init(&dirnames);\n \n \tfor_each_string_list_item(item, refnames) {\n+\t\tconst size_t *update_idx = (size_t *)item->util;\n \t\tconst char *refname = item->string;\n \t\tconst char *extra_refname;\n \t\tstruct object_id oid;\n@@ -2597,14 +2601,30 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs\n \t\t\t\tcontinue;\n \n \t\t\tif (!initial_transaction &&\n-\t\t\t !refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n-\t\t\t\t\t &type, &ignore_errno)) {\n+\t\t\t (strset_contains(&conflicting_dirnames, dirname.buf) ||\n+\t\t\t !refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n+\t\t\t\t\t\t &type, &ignore_errno))) {\n+\t\t\t\tif (transaction && ref_transaction_maybe_set_rejected(\n+\t\t\t\t\t transaction, *update_idx,\n+\t\t\t\t\t REF_TRANSACTION_ERROR_NAME_CONFLICT)) {\n+\t\t\t\t\tstrset_remove(&dirnames, dirname.buf);\n+\t\t\t\t\tstrset_add(&conflicting_dirnames, dirname.buf);\n+\t\t\t\t\tcontinue;\n+\t\t\t\t}\n+\n \t\t\t\tstrbuf_addf(err, _(\"'%s' exists; cannot create '%s'\"),\n \t\t\t\t\t dirname.buf, refname);\n \t\t\t\tgoto cleanup;\n \t\t\t}\n \n \t\t\tif (extras && string_list_has_string(extras, dirname.buf)) {\n+\t\t\t\tif (transaction && ref_transaction_maybe_set_rejected(\n+\t\t\t\t\t transaction, *update_idx,\n+\t\t\t\t\t REF_TRANSACTION_ERROR_NAME_CONFLICT)) {\n+\t\t\t\t\tstrset_remove(&dirnames, dirname.buf);\n+\t\t\t\t\tcontinue;\n+\t\t\t\t}\n+\n \t\t\t\tstrbuf_addf(err, _(\"cannot process '%s' and '%s' at the same time\"),\n \t\t\t\t\t refname, dirname.buf);\n \t\t\t\tgoto cleanup;\n@@ -2637,6 +2657,11 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs\n \t\t\t\t string_list_has_string(skip, iter->refname))\n \t\t\t\t\tcontinue;\n \n+\t\t\t\tif (transaction && ref_transaction_maybe_set_rejected(\n+\t\t\t\t\t transaction, *update_idx,\n+\t\t\t\t\t REF_TRANSACTION_ERROR_NAME_CONFLICT))\n+\t\t\t\t\tcontinue;\n+\n \t\t\t\tstrbuf_addf(err, _(\"'%s' exists; cannot create '%s'\"),\n \t\t\t\t\t iter->refname, refname);\n \t\t\t\tgoto cleanup;\n@@ -2648,6 +2673,11 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs\n \n \t\textra_refname = find_descendant_ref(dirname.buf, extras, skip);\n \t\tif (extra_refname) {\n+\t\t\tif (transaction && ref_transaction_maybe_set_rejected(\n+\t\t\t\t transaction, *update_idx,\n+\t\t\t\t REF_TRANSACTION_ERROR_NAME_CONFLICT))\n+\t\t\t\tcontinue;\n+\n \t\t\tstrbuf_addf(err, _(\"cannot process '%s' and '%s' at the same time\"),\n \t\t\t\t refname, extra_refname);\n \t\t\tgoto cleanup;\n@@ -2659,6 +2689,7 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs\n cleanup:\n \tstrbuf_release(&referent);\n \tstrbuf_release(&dirname);\n+\tstrset_clear(&conflicting_dirnames);\n \tstrset_clear(&dirnames);\n \tref_iterator_free(iter);\n \treturn ret;\n@@ -2679,7 +2710,7 @@ enum ref_transaction_error refs_verify_refname_available(\n \t};\n \n \treturn refs_verify_refnames_available(refs, &refnames, extras, skip,\n-\t\t\t\t\t initial_transaction, err);\n+\t\t\t\t\t NULL, initial_transaction, err);\n }\n \n struct do_for_each_reflog_help {\ndiff --git a/refs.h b/refs.h\nindex c48c800478..46a6008e07 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -141,18 +141,6 @@ enum ref_transaction_error refs_verify_refname_available(struct ref_store *refs,\n \t\t\t\t\t\t unsigned int initial_transaction,\n \t\t\t\t\t\t struct strbuf *err);\n \n-/*\n- * Same as `refs_verify_refname_available()`, but checking for a list of\n- * refnames instead of only a single item. This is more efficient in the case\n- * where one needs to check multiple refnames.\n- */\n-enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,\n-\t\t\t\t\t const struct string_list *refnames,\n-\t\t\t\t\t const struct string_list *extras,\n-\t\t\t\t\t const struct string_list *skip,\n-\t\t\t\t\t unsigned int initial_transaction,\n-\t\t\t\t\t struct strbuf *err);\n-\n int refs_ref_exists(struct ref_store *refs, const char *refname);\n \n int should_autocreate_reflog(enum log_refs_config log_all_ref_updates,\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 256c69b942..b96a511977 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -677,16 +677,18 @@ static void unlock_ref(struct ref_lock *lock)\n * - Generate informative error messages in the case of failure\n */\n static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,\n-\t\t\t\t\t const char *refname,\n+\t\t\t\t\t struct ref_update *update,\n+\t\t\t\t\t size_t update_idx,\n \t\t\t\t\t int mustexist,\n \t\t\t\t\t struct string_list *refnames_to_check,\n \t\t\t\t\t const struct string_list *extras,\n \t\t\t\t\t struct ref_lock **lock_p,\n \t\t\t\t\t struct strbuf *referent,\n-\t\t\t\t\t unsigned int *type,\n \t\t\t\t\t struct strbuf *err)\n {\n \tenum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;\n+\tconst char *refname = update->refname;\n+\tunsigned int *type = &update->type;\n \tstruct ref_lock *lock;\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tint attempts_remaining = 3;\n@@ -785,6 +787,8 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,\n \n \tif (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,\n \t\t\t type, &failure_errno)) {\n+\t\tstruct string_list_item *item;\n+\n \t\tif (failure_errno == ENOENT) {\n \t\t\tif (mustexist) {\n \t\t\t\t/* Garden variety missing reference. */\n@@ -864,7 +868,9 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,\n \t\t * make sure there is no existing packed ref that conflicts\n \t\t * with refname. This check is deferred so that we can batch it.\n \t\t */\n-\t\tstring_list_append(refnames_to_check, refname);\n+\t\titem = string_list_append(refnames_to_check, refname);\n+\t\titem->util = xmalloc(sizeof(update_idx));\n+\t\tmemcpy(item->util, &update_idx, sizeof(update_idx));\n \t}\n \n \tret = 0;\n@@ -2547,6 +2553,7 @@ struct files_transaction_backend_data {\n */\n static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *refs,\n \t\t\t\t\t\t struct ref_update *update,\n+\t\t\t\t\t\t size_t update_idx,\n \t\t\t\t\t\t struct ref_transaction *transaction,\n \t\t\t\t\t\t const char *head_ref,\n \t\t\t\t\t\t struct string_list *refnames_to_check,\n@@ -2575,9 +2582,9 @@ static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *re\n \tif (lock) {\n \t\tlock->count++;\n \t} else {\n-\t\tret = lock_raw_ref(refs, update->refname, mustexist,\n+\t\tret = lock_raw_ref(refs, update, update_idx, mustexist,\n \t\t\t\t refnames_to_check, &transaction->refnames,\n-\t\t\t\t &lock, &referent, &update->type, err);\n+\t\t\t\t &lock, &referent, err);\n \t\tif (ret) {\n \t\t\tchar *reason;\n \n@@ -2849,7 +2856,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n \n-\t\tret = lock_ref_for_update(refs, update, transaction,\n+\t\tret = lock_ref_for_update(refs, update, i, transaction,\n \t\t\t\t\t head_ref, &refnames_to_check,\n \t\t\t\t\t err);\n \t\tif (ret) {\n@@ -2905,7 +2912,8 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \t * So instead, we accept the race for now.\n \t */\n \tif (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,\n-\t\t\t\t\t &transaction->refnames, NULL, 0, err)) {\n+\t\t\t\t\t &transaction->refnames, NULL, transaction,\n+\t\t\t\t\t 0, err)) {\n \t\tret = REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t\tgoto cleanup;\n \t}\n@@ -2951,7 +2959,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,\n \n cleanup:\n \tfree(head_ref);\n-\tstring_list_clear(&refnames_to_check, 0);\n+\tstring_list_clear(&refnames_to_check, 1);\n \n \tif (ret)\n \t\tfiles_transaction_cleanup(refs, transaction);\n@@ -3097,7 +3105,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \t}\n \n \tif (refs_verify_refnames_available(&refs->base, &refnames_to_check,\n-\t\t\t\t\t &affected_refnames, NULL, 1, err)) {\n+\t\t\t\t\t &affected_refnames, NULL, transaction,\n+\t\t\t\t\t 1, err)) {\n \t\tpacked_refs_unlock(refs->packed_ref_store);\n \t\tret = REF_TRANSACTION_ERROR_NAME_CONFLICT;\n \t\tgoto cleanup;\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 73a5379b73..f868870851 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -806,4 +806,20 @@ enum ref_transaction_error ref_update_check_old_target(const char *referent,\n */\n int ref_update_expects_existing_old_ref(struct ref_update *update);\n \n+/*\n+ * Same as `refs_verify_refname_available()`, but checking for a list of\n+ * refnames instead of only a single item. This is more efficient in the case\n+ * where one needs to check multiple refnames.\n+ *\n+ * If using batched updates, then individual updates are marked rejected,\n+ * reference backends are then in charge of not committing those updates.\n+ */\n+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,\n+\t\t\t\t\t const struct string_list *refnames,\n+\t\t\t\t\t const struct string_list *extras,\n+\t\t\t\t\t const struct string_list *skip,\n+\t\t\t\t\t struct ref_transaction *transaction,\n+\t\t\t\t\t unsigned int initial_transaction,\n+\t\t\t\t\t struct strbuf *err);\n+\n #endif /* REFS_REFS_INTERNAL_H */\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex 5db4a108b9..4c3817f4ec 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -1074,6 +1074,7 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor\n \t\t\t\t\t\t\tstruct ref_transaction *transaction,\n \t\t\t\t\t\t\tstruct reftable_backend *be,\n \t\t\t\t\t\t\tstruct ref_update *u,\n+\t\t\t\t\t\t\tsize_t update_idx,\n \t\t\t\t\t\t\tstruct string_list *refnames_to_check,\n \t\t\t\t\t\t\tunsigned int head_type,\n \t\t\t\t\t\t\tstruct strbuf *head_referent,\n@@ -1149,6 +1150,7 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor\n \tif (ret < 0)\n \t\treturn REF_TRANSACTION_ERROR_GENERIC;\n \tif (ret > 0 && !ref_update_expects_existing_old_ref(u)) {\n+\t\tstruct string_list_item *item;\n \t\t/*\n \t\t * The reference does not exist, and we either have no\n \t\t * old object ID or expect the reference to not exist.\n@@ -1158,7 +1160,9 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor\n \t\t * can output a proper error message instead of failing\n \t\t * at a later point.\n \t\t */\n-\t\tstring_list_append(refnames_to_check, u->refname);\n+\t\titem = string_list_append(refnames_to_check, u->refname);\n+\t\titem->util = xmalloc(sizeof(update_idx));\n+\t\tmemcpy(item->util, &update_idx, sizeof(update_idx));\n \n \t\t/*\n \t\t * There is no need to write the reference deletion\n@@ -1368,7 +1372,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tret = prepare_single_update(refs, tx_data, transaction, be,\n-\t\t\t\t\t transaction->updates[i],\n+\t\t\t\t\t transaction->updates[i], i,\n \t\t\t\t\t &refnames_to_check, head_type,\n \t\t\t\t\t &head_referent, &referent, err);\n \t\tif (ret) {\n@@ -1384,6 +1388,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \n \tret = refs_verify_refnames_available(ref_store, &refnames_to_check,\n \t\t\t\t\t &transaction->refnames, NULL,\n+\t\t\t\t\t transaction,\n \t\t\t\t\t transaction->flags & REF_TRANSACTION_FLAG_INITIAL,\n \t\t\t\t\t err);\n \tif (ret < 0)\n@@ -1402,7 +1407,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,\n \t}\n \tstrbuf_release(&referent);\n \tstrbuf_release(&head_referent);\n-\tstring_list_clear(&refnames_to_check, 0);\n+\tstring_list_clear(&refnames_to_check, 1);\n \n \treturn ret;\n }\n-- \n2.48.1\n\n"},{"id":"515843","messageId":"20250408085120.614893-9-karthik.188@gmail.com","threadId":"62915","inReplyTo":"20250408085120.614893-1-karthik.188@gmail.com","subject":"[PATCH v6 8/8] update-ref: add --batch-updates flag for stdin mode","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-08T08:51:12Z","receivedAt":"2025-04-08T08:51:33Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"When updating multiple references through stdin, Git's update-ref\ncommand normally aborts the entire transaction if any single update\nfails. This atomic behavior prevents partial updates. Introduce a new\nbatch update system, where the updates the performed together similar\nbut individual updates are allowed to fail.\n\nAdd a new `--batch-updates` flag that allows the transaction to continue\neven when individual reference updates fail. This flag can only be used\nin `--stdin` mode and builds upon the batch update support added to the\nrefs subsystem in the previous commits. When enabled, failed updates are\nreported in the following format:\n\n rejected SP ( | ) SP ( | ) SP LF\n\nUpdate the documentation to reflect this change and also tests to cover\ndifferent scenarios where an update could be rejected.\n\nSigned-off-by: Karthik Nayak \n---\n Documentation/git-update-ref.adoc | 14 +-\n builtin/update-ref.c | 66 ++++++++-\n t/t1400-update-ref.sh | 233 ++++++++++++++++++++++++++++++\n 3 files changed, 306 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/git-update-ref.adoc b/Documentation/git-update-ref.adoc\nindex 9e6935d38d..9310ce9768 100644\n--- a/Documentation/git-update-ref.adoc\n+++ b/Documentation/git-update-ref.adoc\n@@ -7,8 +7,10 @@ git-update-ref - Update the object name stored in a ref safely\n \n SYNOPSIS\n --------\n-[verse]\n-'git update-ref' [-m ] [--no-deref] (-d [] | [--create-reflog] [] | --stdin [-z])\n+[synopsis]\n+git update-ref [-m ] [--no-deref] -d []\n+git update-ref [-m ] [--no-deref] [--create-reflog] []\n+git update-ref [-m ] [--no-deref] --stdin [-z] [--batch-updates]\n \n DESCRIPTION\n -----------\n@@ -57,6 +59,14 @@ performs all modifications together. Specify commands of the form:\n With `--create-reflog`, update-ref will create a reflog for each ref\n even if one would not ordinarily be created.\n \n+With `--batch-updates`, update-ref executes the updates in a batch but allows\n+individual updates to fail due to invalid or incorrect user input, applying only\n+the successful updates. However, system-related errors—such as I/O failures or\n+memory issues—will result in a full failure of all batched updates. Any failed\n+updates will be reported in the following format:\n+\n+\trejected SP ( | ) SP ( | ) SP LF\n+\n Quote fields containing whitespace as if they were strings in C source\n code; i.e., surrounded by double-quotes and with backslash escapes.\n Use 40 \"0\" characters or the empty string to specify a zero value. To\ndiff --git a/builtin/update-ref.c b/builtin/update-ref.c\nindex 1d541e13ad..111d6473ad 100644\n--- a/builtin/update-ref.c\n+++ b/builtin/update-ref.c\n@@ -5,6 +5,7 @@\n #include \"config.h\"\n #include \"gettext.h\"\n #include \"hash.h\"\n+#include \"hex.h\"\n #include \"refs.h\"\n #include \"object-name.h\"\n #include \"parse-options.h\"\n@@ -13,7 +14,7 @@\n static const char * const git_update_ref_usage[] = {\n \tN_(\"git update-ref [] -d []\"),\n \tN_(\"git update-ref [] []\"),\n-\tN_(\"git update-ref [] --stdin [-z]\"),\n+\tN_(\"git update-ref [] --stdin [-z] [--batch-updates]\"),\n \tNULL\n };\n \n@@ -565,6 +566,49 @@ static void parse_cmd_abort(struct ref_transaction *transaction,\n \treport_ok(\"abort\");\n }\n \n+static void print_rejected_refs(const char *refname,\n+\t\t\t\tconst struct object_id *old_oid,\n+\t\t\t\tconst struct object_id *new_oid,\n+\t\t\t\tconst char *old_target,\n+\t\t\t\tconst char *new_target,\n+\t\t\t\tenum ref_transaction_error err,\n+\t\t\t\tvoid *cb_data UNUSED)\n+{\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tconst char *reason = \"\";\n+\n+\tswitch (err) {\n+\tcase REF_TRANSACTION_ERROR_NAME_CONFLICT:\n+\t\treason = \"refname conflict\";\n+\t\tbreak;\n+\tcase REF_TRANSACTION_ERROR_CREATE_EXISTS:\n+\t\treason = \"reference already exists\";\n+\t\tbreak;\n+\tcase REF_TRANSACTION_ERROR_NONEXISTENT_REF:\n+\t\treason = \"reference does not exist\";\n+\t\tbreak;\n+\tcase REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE:\n+\t\treason = \"incorrect old value provided\";\n+\t\tbreak;\n+\tcase REF_TRANSACTION_ERROR_INVALID_NEW_VALUE:\n+\t\treason = \"invalid new value provided\";\n+\t\tbreak;\n+\tcase REF_TRANSACTION_ERROR_EXPECTED_SYMREF:\n+\t\treason = \"expected symref but found regular ref\";\n+\t\tbreak;\n+\tdefault:\n+\t\treason = \"unkown failure\";\n+\t}\n+\n+\tstrbuf_addf(&sb, \"rejected %s %s %s %s\\n\", refname,\n+\t\t new_oid ? oid_to_hex(new_oid) : new_target,\n+\t\t old_oid ? oid_to_hex(old_oid) : old_target,\n+\t\t reason);\n+\n+\tfwrite(sb.buf, sb.len, 1, stdout);\n+\tstrbuf_release(&sb);\n+}\n+\n static void parse_cmd_commit(struct ref_transaction *transaction,\n \t\t\t const char *next, const char *end UNUSED)\n {\n@@ -573,6 +617,10 @@ static void parse_cmd_commit(struct ref_transaction *transaction,\n \t\tdie(\"commit: extra input: %s\", next);\n \tif (ref_transaction_commit(transaction, &error))\n \t\tdie(\"commit: %s\", error.buf);\n+\n+\tref_transaction_for_each_rejected_update(transaction,\n+\t\t\t\t\t\t print_rejected_refs, NULL);\n+\n \treport_ok(\"commit\");\n \tref_transaction_free(transaction);\n }\n@@ -609,7 +657,7 @@ static const struct parse_cmd {\n \t{ \"commit\", parse_cmd_commit, 0, UPDATE_REFS_CLOSED },\n };\n \n-static void update_refs_stdin(void)\n+static void update_refs_stdin(unsigned int flags)\n {\n \tstruct strbuf input = STRBUF_INIT, err = STRBUF_INIT;\n \tenum update_refs_state state = UPDATE_REFS_OPEN;\n@@ -617,7 +665,7 @@ static void update_refs_stdin(void)\n \tint i, j;\n \n \ttransaction = ref_store_transaction_begin(get_main_ref_store(the_repository),\n-\t\t\t\t\t\t 0, &err);\n+\t\t\t\t\t\t flags, &err);\n \tif (!transaction)\n \t\tdie(\"%s\", err.buf);\n \n@@ -685,7 +733,7 @@ static void update_refs_stdin(void)\n \t\t\t */\n \t\t\tstate = cmd->state;\n \t\t\ttransaction = ref_store_transaction_begin(get_main_ref_store(the_repository),\n-\t\t\t\t\t\t\t\t 0, &err);\n+\t\t\t\t\t\t\t\t flags, &err);\n \t\t\tif (!transaction)\n \t\t\t\tdie(\"%s\", err.buf);\n \n@@ -701,6 +749,8 @@ static void update_refs_stdin(void)\n \t\t/* Commit by default if no transaction was requested. */\n \t\tif (ref_transaction_commit(transaction, &err))\n \t\t\tdie(\"%s\", err.buf);\n+\t\tref_transaction_for_each_rejected_update(transaction,\n+\t\t\t\t\t\t print_rejected_refs, NULL);\n \t\tref_transaction_free(transaction);\n \t\tbreak;\n \tcase UPDATE_REFS_STARTED:\n@@ -727,6 +777,8 @@ int cmd_update_ref(int argc,\n \tstruct object_id oid, oldoid;\n \tint delete = 0, no_deref = 0, read_stdin = 0, end_null = 0;\n \tint create_reflog = 0;\n+\tunsigned int flags = 0;\n+\n \tstruct option options[] = {\n \t\tOPT_STRING( 'm', NULL, &msg, N_(\"reason\"), N_(\"reason of the update\")),\n \t\tOPT_BOOL('d', NULL, &delete, N_(\"delete the reference\")),\n@@ -735,6 +787,8 @@ int cmd_update_ref(int argc,\n \t\tOPT_BOOL('z', NULL, &end_null, N_(\"stdin has NUL-terminated arguments\")),\n \t\tOPT_BOOL( 0 , \"stdin\", &read_stdin, N_(\"read updates from stdin\")),\n \t\tOPT_BOOL( 0 , \"create-reflog\", &create_reflog, N_(\"create a reflog\")),\n+\t\tOPT_BIT('0', \"batch-updates\", &flags, N_(\"batch reference updates\"),\n+\t\t\tREF_TRANSACTION_ALLOW_FAILURE),\n \t\tOPT_END(),\n \t};\n \n@@ -756,8 +810,10 @@ int cmd_update_ref(int argc,\n \t\t\tusage_with_options(git_update_ref_usage, options);\n \t\tif (end_null)\n \t\t\tline_termination = '\\0';\n-\t\tupdate_refs_stdin();\n+\t\tupdate_refs_stdin(flags);\n \t\treturn 0;\n+\t} else if (flags & REF_TRANSACTION_ALLOW_FAILURE) {\n+\t\tdie(\"--batch-updates can only be used with --stdin\");\n \t}\n \n \tif (end_null)\ndiff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh\nindex 29045aad43..d29d23cb89 100755\n--- a/t/t1400-update-ref.sh\n+++ b/t/t1400-update-ref.sh\n@@ -2066,6 +2066,239 @@ do\n \t\tgrep \"$(git rev-parse $a) $(git rev-parse $a)\" actual\n \t'\n \n+\ttest_expect_success \"stdin $type batch-updates\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit commit &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$head\" \"$Z\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$head\" \"$Z\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --batch-updates expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\tgit rev-parse refs/heads/ref2 >actual &&\n+\t\t\ttest_cmp expect actual\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type batch-updates with invalid new_oid\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\t\t\tgit update-ref refs/heads/ref2 $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$(test_oid 001)\" \"$head\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --batch-updates stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\techo $head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref2 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"invalid new value provided\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type batch-updates with non-commit new_oid\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\thead_tree=$(git rev-parse HEAD^{tree}) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\t\t\tgit update-ref refs/heads/ref2 $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$head_tree\" \"$head\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --batch-updates stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\techo $head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref2 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"invalid new value provided\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type batch-updates with non-existent ref\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$old_head\" \"$head\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --batch-updates stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_must_fail git rev-parse refs/heads/ref2 &&\n+\t\t\ttest_grep -q \"reference does not exist\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type batch-updates with dangling symref\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\t\t\tgit symbolic-ref refs/heads/ref2 refs/heads/nonexistent &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$old_head\" \"$head\" >>stdin &&\n+\t\t\tgit update-ref $type --no-deref --stdin --batch-updates stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\techo $head >expect &&\n+\t\t\ttest_must_fail git rev-parse refs/heads/ref2 &&\n+\t\t\ttest_grep -q \"reference does not exist\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type batch-updates with regular ref as symref\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\t\t\tgit update-ref refs/heads/ref2 $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"symref-update refs/heads/ref2\" \"$old_head\" \"ref\" \"refs/heads/nonexistent\" >>stdin &&\n+\t\t\tgit update-ref $type --no-deref --stdin --batch-updates stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\techo $head >expect &&\n+\t\t\techo $head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref2 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"expected symref but found regular ref\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type batch-updates with invalid old_oid\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\t\t\tgit update-ref refs/heads/ref2 $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$old_head\" \"$Z\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --batch-updates stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\techo $head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref2 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"reference already exists\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type batch-updates with incorrect old oid\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref1 $head &&\n+\t\t\tgit update-ref refs/heads/ref2 $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref1\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref2\" \"$head\" \"$old_head\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --batch-updates stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref1 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\techo $head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref2 >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"incorrect old value provided\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type batch-updates refname conflict\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref/foo $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/ref/foo\" \"$old_head\" \"$head\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref\" \"$old_head\" \"\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --batch-updates stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/ref/foo >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"refname conflict\" stdout\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"stdin $type batch-updates refname conflict new ref\" '\n+\t\tgit init repo &&\n+\t\ttest_when_finished \"rm -fr repo\" &&\n+\t\t(\n+\t\t\tcd repo &&\n+\t\t\ttest_commit one &&\n+\t\t\told_head=$(git rev-parse HEAD) &&\n+\t\t\ttest_commit two &&\n+\t\t\thead=$(git rev-parse HEAD) &&\n+\t\t\tgit update-ref refs/heads/ref/foo $head &&\n+\n+\t\t\tformat_command $type \"update refs/heads/foo\" \"$old_head\" \"\" >stdin &&\n+\t\t\tformat_command $type \"update refs/heads/ref\" \"$old_head\" \"\" >>stdin &&\n+\t\t\tgit update-ref $type --stdin --batch-updates stdout &&\n+\t\t\techo $old_head >expect &&\n+\t\t\tgit rev-parse refs/heads/foo >actual &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\ttest_grep -q \"refname conflict\" stdout\n+\t\t)\n+\t'\n done\n \n test_expect_success 'update-ref should also create reflog for HEAD' '\n-- \n2.48.1\n\n"},{"id":"515877","messageId":"xmqqfriiy9vh.fsf@gitster.g","threadId":"62915","inReplyTo":"20250408085120.614893-9-karthik.188@gmail.com","subject":"Re: [PATCH v6 8/8] update-ref: add --batch-updates flag for stdin mode","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-04-08T15:02:26Z","receivedAt":"2025-04-08T15:02:29Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Karthik Nayak writes:\n\n> Content-Type: text/plain; charset=y\n\nPlease don't ;-).\n\nMore practically, is there something we can do to avoid this\nhappening in send-email? It may be a not-so-uncommon end user\nmistake that we would rather help our users avoid.\n\n> When updating multiple references through stdin, Git's update-ref\n> ...\n\nWill replace and queue. Let me mark the topic for 'next'.\n"},{"id":"515878","messageId":"CAOLa=ZT4fOz1sd31EhyLGeyQzS6GO0aSYN0L4fdPALRixkD6xQ@mail.gmail.com","threadId":"62915","inReplyTo":"xmqqfriiy9vh.fsf@gitster.g","subject":"Re: [PATCH v6 8/8] update-ref: add --batch-updates flag for stdin mode","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-08T15:26:28Z","receivedAt":"2025-04-08T15:26:32Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Junio C Hamano writes:\n\n> Karthik Nayak writes:\n>\n>> Content-Type: text/plain; charset=y\n>\n> Please don't ;-).\n>\n> More practically, is there something we can do to avoid this\n> happening in send-email? It may be a not-so-uncommon end user\n> mistake that we would rather help our users avoid.\n>\n\nThis seems like this was in response to the following question:\n Which 8bit encoding should I declare [UTF-8]?\n\nWhich I should have just clicked 'Enter' on, but typed 'y' as 'yes\nplease pick UTF-8'. Which again confirms the encoding, which I\npresumably didn't read. So I guess the problem Exists Between Keyboard\nand Chair.\n\n>> When updating multiple references through stdin, Git's update-ref\n>> ...\n>\n> Will replace and queue. Let me mark the topic for 'next'.\n\nThanks!\n"},{"id":"515892","messageId":"xmqq7c3uy2ps.fsf@gitster.g","threadId":"62915","inReplyTo":"CAOLa=ZT4fOz1sd31EhyLGeyQzS6GO0aSYN0L4fdPALRixkD6xQ@mail.gmail.com","subject":"Re: [PATCH v6 8/8] update-ref: add --batch-updates flag for stdin mode","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-04-08T17:37:03Z","receivedAt":"2025-04-08T17:37:06Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Karthik Nayak writes:\n\n> Junio C Hamano writes:\n>\n>> Karthik Nayak writes:\n>>\n>>> Content-Type: text/plain; charset=y\n>>\n>> Please don't ;-).\n>>\n>> More practically, is there something we can do to avoid this\n>> happening in send-email? It may be a not-so-uncommon end user\n>> mistake that we would rather help our users avoid.\n>>\n>\n> This seems like this was in response to the following question:\n> Which 8bit encoding should I declare [UTF-8]?\n>\n> Which I should have just clicked 'Enter' on, but typed 'y' as 'yes\n> please pick UTF-8'. Which again confirms the encoding, which I\n> presumably didn't read. So I guess the problem Exists Between Keyboard\n> and Chair.\n\nOK, we have seen enough people got burned by 'y', and made 852a15d7\n(send-email: ask confirmation if given encoding name is very short,\n2015-02-13) as a response exactly for that problem, but it is not\neffective as we wished X-<.\n\nIf there were a better validation method than \"4 bytes or longer\" we\ncurrently use for valid values for \"charset=$auto_8bit_encoding\", we\ncould lose confirm_only from the call to ask() that asks the\nquestion, but I do not know if that is feasible.\n\nAnother more obvious alternative is to do something ugly like this\npatch, I suppose? Just like is taken as \"I take the default\nvalue presented\", this makes yes mean the same thing.\n\nThere is one question that asks yes/no question with default set to\n'n', which would be broken by the patch below, so it needs a bit\nmore thought, though.\n\n git-send-email.perl | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git c/git-send-email.perl w/git-send-email.perl\nindex 798d59b84f..8b942e5bcf 100755\n--- c/git-send-email.perl\n+++ w/git-send-email.perl\n@@ -986,7 +986,8 @@ sub ask {\n \t\t\tprint \"\\n\";\n \t\t\treturn defined $default ? $default : undef;\n \t\t}\n-\t\tif ($resp eq '' and defined $default) {\n+\t\tif (defined $default &&\n+\t\t ($resp eq '' || $resp =~ /^y(?:es)$/i)) {\n \t\t\treturn $default;\n \t\t}\n \t\tif (!defined $valid_re or $resp =~ /$valid_re/) {\n\n\n"},{"id":"515947","messageId":"CAOLa=ZQ2eTWbj18+QPp=pTAovPVAQVYwJC6_ZTuMvCU-oQOToA@mail.gmail.com","threadId":"62915","inReplyTo":"xmqq7c3uy2ps.fsf@gitster.g","subject":"Re: [PATCH v6 8/8] update-ref: add --batch-updates flag for stdin mode","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-10T11:23:00Z","receivedAt":"2025-04-10T11:23:02Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Junio C Hamano writes:\n\n> Karthik Nayak writes:\n>\n>> Junio C Hamano writes:\n>>\n>>> Karthik Nayak writes:\n>>>\n>>>> Content-Type: text/plain; charset=y\n>>>\n>>> Please don't ;-).\n>>>\n>>> More practically, is there something we can do to avoid this\n>>> happening in send-email? It may be a not-so-uncommon end user\n>>> mistake that we would rather help our users avoid.\n>>>\n>>\n>> This seems like this was in response to the following question:\n>> Which 8bit encoding should I declare [UTF-8]?\n>>\n>> Which I should have just clicked 'Enter' on, but typed 'y' as 'yes\n>> please pick UTF-8'. Which again confirms the encoding, which I\n>> presumably didn't read. So I guess the problem Exists Between Keyboard\n>> and Chair.\n>\n> OK, we have seen enough people got burned by 'y', and made 852a15d7\n> (send-email: ask confirmation if given encoding name is very short,\n> 2015-02-13) as a response exactly for that problem, but it is not\n> effective as we wished X-<.\n>\n\nI see.\n\n> If there were a better validation method than \"4 bytes or longer\" we\n> currently use for valid values for \"charset=$auto_8bit_encoding\", we\n> could lose confirm_only from the call to ask() that asks the\n> question, but I do not know if that is feasible.\n>\n> Another more obvious alternative is to do something ugly like this\n> patch, I suppose? Just like is taken as \"I take the default\n> value presented\", this makes yes mean the same thing.\n>\n> There is one question that asks yes/no question with default set to\n> 'n', which would be broken by the patch below, so it needs a bit\n> more thought, though.\n>\n\nYes, this would be an issue. I think what would be nice is perhaps an\noption like $yes_default.\n\n> git-send-email.perl | 3 ++-\n> 1 file changed, 2 insertions(+), 1 deletion(-)\n>\n> diff --git c/git-send-email.perl w/git-send-email.perl\n> index 798d59b84f..8b942e5bcf 100755\n> --- c/git-send-email.perl\n> +++ w/git-send-email.perl\n> @@ -986,7 +986,8 @@ sub ask {\n> \t\t\tprint \"\\n\";\n> \t\t\treturn defined $default ? $default : undef;\n> \t\t}\n> -\t\tif ($resp eq '' and defined $default) {\n> +\t\tif (defined $default &&\n> +\t\t ($resp eq '' || $resp =~ /^y(?:es)$/i)) {\n> \t\t\treturn $default;\n> \t\t}\n> \t\tif (!defined $valid_re or $resp =~ /$valid_re/) {\n\nGoing on top of your patch, something like:\n\n-->8--\n\ndiff --git a/git-send-email.perl b/git-send-email.perl\nindex 798d59b84f..318699d26c 100755\n--- a/git-send-email.perl\n+++ b/git-send-email.perl\n@@ -974,6 +974,7 @@ sub ask {\n \tmy $valid_re = $arg{valid_re};\n \tmy $default = $arg{default};\n \tmy $confirm_only = $arg{confirm_only};\n+\tmy $yes_is_default = $arg{yes_is_default};\n \tmy $resp;\n \tmy $i = 0;\n \tmy $term = term();\n@@ -989,6 +990,10 @@ sub ask {\n \t\tif ($resp eq '' and defined $default) {\n \t\t\treturn $default;\n \t\t}\n+\t\tif (defined $default and defined $yes_is_default\n+\t\t and $resp =~ /^y(?:es)$/i) {\n+\t\t\treturn $default;\n+\t\t}\n \t\tif (!defined $valid_re or $resp =~ /$valid_re/) {\n \t\t\treturn $resp;\n \t\t}\n@@ -1031,7 +1036,7 @@ sub file_declares_8bit_cte {\n \t}\n \t$auto_8bit_encoding = ask(__(\"Which 8bit encoding should I declare\n[UTF-8]? \"),\n \t\t\t\t valid_re => qr/.{4}/, confirm_only => 1,\n-\t\t\t\t default => \"UTF-8\");\n+\t\t\t\t default => \"UTF-8\", yes_is_default => true);\n }\n\n if (!$force) {\n"}]}