git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v10 25/44] receive-pack.c: use a reference transaction for updating the refs

From
Ronnie Sahlberg <sahlberg@google.com>
Date
May 19, 2014, 19:02 UTC
Message-ID
<CAL=YDWmLgW0b28q5Yqw7R4nobKF5=pcbSpnazC8+EA=QKhkpow@mail.gmail.com>
In-Reply-To
<537781CA.1010208@alum.mit.edu>
On Sat, May 17, 2014 at 8:35 AM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
Show 73 quoted lines
> On 05/16/2014 07:37 PM, Ronnie Sahlberg wrote:
>> Wrap all the ref updates inside a transaction to make the update atomic.
>>
>> Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
>> ---
>>  builtin/receive-pack.c | 20 ++++++++++----------
>>  1 file changed, 10 insertions(+), 10 deletions(-)
>>
>> diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
>> index c323081..5534138 100644
>> --- a/builtin/receive-pack.c
>> +++ b/builtin/receive-pack.c
>> @@ -46,6 +46,8 @@ static void *head_name_to_free;
>>  static int sent_capabilities;
>>  static int shallow_update;
>>  static const char *alt_shallow_file;
>> +static struct strbuf err = STRBUF_INIT;
>> +static struct ref_transaction *transaction;
>>
>>  static enum deny_action parse_deny_action(const char *var, const char *value)
>>  {
>> @@ -475,7 +477,6 @@ static const char *update(struct command *cmd, struct shallow_info *si)
>>       const char *namespaced_name;
>>       unsigned char *old_sha1 = cmd->old_sha1;
>>       unsigned char *new_sha1 = cmd->new_sha1;
>> -     struct ref_lock *lock;
>>
>>       /* only refs/... are allowed */
>>       if (!starts_with(name, "refs/") || check_refname_format(name + 5, 0)) {
>> @@ -580,15 +581,9 @@ static const char *update(struct command *cmd, struct shallow_info *si)
>>                   update_shallow_ref(cmd, si))
>>                       return "shallow error";
>>
>> -             lock = lock_any_ref_for_update(namespaced_name, old_sha1,
>> -                                            0, NULL);
>> -             if (!lock) {
>> -                     rp_error("failed to lock %s", name);
>> -                     return "failed to lock";
>> -             }
>> -             if (write_ref_sha1(lock, new_sha1, "push")) {
>> -                     return "failed to write"; /* error() already called */
>> -             }
>> +             if (ref_transaction_update(transaction, namespaced_name,
>> +                                        new_sha1, old_sha1, 0, 1, &err))
>> +                     return "failed to update";
>>               return NULL; /* good */
>>       }
>>  }
>> @@ -812,6 +807,7 @@ static void execute_commands(struct command *commands,
>>       head_name = head_name_to_free = resolve_refdup("HEAD", sha1, 0, NULL);
>>
>>       checked_connectivity = 1;
>> +     transaction = ref_transaction_begin();
>>       for (cmd = commands; cmd; cmd = cmd->next) {
>>               if (cmd->error_string)
>>                       continue;
>> @@ -827,6 +823,10 @@ static void execute_commands(struct command *commands,
>>                       checked_connectivity = 0;
>>               }
>>       }
>> +     if (ref_transaction_commit(transaction, "push", &err))
>> +             error("%s", err.buf);
>> +     ref_transaction_free(transaction);
>> +     strbuf_release(&err);
>>
>>       if (shallow_update && !checked_connectivity)
>>               error("BUG: run 'git fsck' for safety.\n"
>>
>
> This patch is strange, because even if one ref_transaction_update() call
> fails, subsequent updates are nevertheless also attempted, and the
> ref_transaction_commit() is also attempted.  Is this an officially
> sanctioned use of the ref_transactions API?  Should it be?

I think it should be supported. Because otherwise, unless you have the entire transaction localized in a single block you would end up having to check and recheck the return value everywhere.

It makes the API much easier to use if you can continue calling transaction functions even after the transaction has failed. If the transaction has already failed then _update/_create/_delete will do nothing except return an error.

If _commit is called on a failed transaction then the commit will fail with an error and do nothing.

I think it is convenient, and it allows things like :
struct ref_transaction *transaction;
void foo()
{
   ...
   ref_transaction_update(transaction, ... , &err);
   ...
}

transaction = ref_transaction_begin(&err); ... doing stuff and call things that eventually ends up calling foo, possible multiple times ... ret = ref_transaction_commit(transaction, &err);

In foo() we ignore checking the return value so we will not see/care if it failed. IF it fails however it will mark the transaction as failed and update &err. (Note that this can not yet happen since _update can not really fail, ever, but the next series will introduce _update failures when we move locking there.)

Instead we can depend on that IF _update failed, then the call to _commit will fail too and &err is already updated so we can defer any checking for errors until _commit time.

This will make the API much more convenient for use cases where you begin/commit the transaction in one function but the calls to _update/_delete/_create are somewhere else, possible many function calls away. It does not mean that a caller must ignore the return value from ref_transaction_update, just that the caller can do so and defer checking for errors until later when it would be more convenient.

Please see current: https://github.com/rsahlberg/git/tree/ref-transactions and patch: refs.c: add transaction.status and track OPEN/CLOSED/ERROR

  It might be
Show 6 quoted lines
> a way to give feedback to the user on multiple attempted reference
> updates at once (i.e., address my comment about the last patch).
>
> If this is sanctioned, then it might be appropriate for the transaction
> to keep track of the fact that one or more reference updates failed, and
> when *_commit() is called to fail the whole transaction.

Yes. I updated refs.h to indicate that you can continue using _update/_create/_delete even if a previous call has failed but that these calls will now just return an error.

This does mean that on the first update that fails for a ref we fail the transaction and abort any further _update calls to fail immediately so if there would be additional refs that would fail we would not log this. I think this is what we want to do since once we have had a ref update fail it would be really hard to determine if the next failure was just a side effect of the first failure or not.

Show 14 quoted lines
>
> In any case, I think it is important to document, as part of the API
> docs, whether this is sanctioned or not, and if so, what exactly are its
> semantics.
>
> I've run out of time for today so I'm going to have to stop here.  FWIW
> patches 01-23 looked OK aside from the comments that I have made.
>
> Michael
>
> --
> Michael Haggerty
> mhagger@alum.mit.edu
> http://softwareswirl.blogspot.com/
Previous: Michael HaggertyNext: Michael Haggerty
Message 41 of 65 in “Use ref transactions for all ref updates”
  1. 00/44 Use ref transactions for all ref updatesRonnie Sahlberg, May 16, 2014
  2. 01/44 refs.c: constify the sha arguments for ref_transaction_create|delete|updateRonnie Sahlberg, May 16, 2014
  3. 02/44 refs.c: allow passing NULL to ref_transaction_freeRonnie Sahlberg, May 16, 2014
  4. 03/44 refs.c: add a strbuf argument to ref_transaction_commit for error loggingRonnie Sahlberg, May 16, 2014
  5. 04/44 refs.c: add an err argument to repack_without_refsRonnie Sahlberg, May 16, 2014
  6. Michael HaggertyMay 17, 2014
  7. Ronnie SahlbergMay 27, 2014
  8. 05/44 refs.c: make ref_update_reject_duplicates take a strbuf argument for errorsRonnie Sahlberg, May 16, 2014
  9. 06/44 refs.c: add an err argument to delete_ref_looseRonnie Sahlberg, May 16, 2014
  10. 07/44 refs.c: make update_ref_write update a strbuf on failureRonnie Sahlberg, May 16, 2014
  11. 08/44 update-ref.c: log transaction error from the update_refRonnie Sahlberg, May 16, 2014
  12. 09/44 refs.c: remove the onerr argument to ref_transaction_commitRonnie Sahlberg, May 16, 2014
  13. 10/44 refs.c: change ref_transaction_update() to do error checking and return statusRonnie Sahlberg, May 16, 2014
  14. 11/44 refs.c: change ref_transaction_create to do error checking and return statusRonnie Sahlberg, May 16, 2014
  15. 12/44 refs.c: ref_transaction_delete to check for error and return statusRonnie Sahlberg, May 16, 2014
  16. 13/44 tag.c: use ref transactions when doing updatesRonnie Sahlberg, May 16, 2014
  17. Michael HaggertyMay 17, 2014
  18. Ronnie SahlbergMay 19, 2014
  19. Ronnie SahlbergMay 19, 2014
  20. 14/44 replace.c: use the ref transaction functions for updatesRonnie Sahlberg, May 16, 2014
  21. Michael HaggertyMay 17, 2014
  22. Ronnie SahlbergMay 19, 2014
  23. 15/44 commit.c: use ref transactions for updatesRonnie Sahlberg, May 16, 2014
  24. 16/44 sequencer.c: use ref transactions for all ref updatesRonnie Sahlberg, May 16, 2014
  25. 17/44 fast-import.c: change update_branch to use ref transactionsRonnie Sahlberg, May 16, 2014
  26. 18/44 branch.c: use ref transaction for all ref updatesRonnie Sahlberg, May 16, 2014
  27. Michael HaggertyMay 17, 2014
  28. Ronnie SahlbergMay 19, 2014
  29. 19/44 refs.c: change update_ref to use a transactionRonnie Sahlberg, May 16, 2014
  30. 20/44 refs.c: free the transaction before returning when number of updates is 0Ronnie Sahlberg, May 16, 2014
  31. 21/44 refs.c: ref_transaction_commit should not free the transactionRonnie Sahlberg, May 16, 2014
  32. 22/44 fetch.c: clear errno before calling functions that might set itRonnie Sahlberg, May 16, 2014
  33. Michael HaggertyMay 17, 2014
  34. Ronnie SahlbergMay 27, 2014
  35. 23/44 fetch.c: change s_update_ref to use a ref transactionRonnie Sahlberg, May 16, 2014
  36. 24/44 fetch.c: use a single ref transaction for all ref updatesRonnie Sahlberg, May 16, 2014
  37. Michael HaggertyMay 17, 2014
  38. Michael HaggertyMay 17, 2014
  39. 25/44 receive-pack.c: use a reference transaction for updating the refsRonnie Sahlberg, May 16, 2014
  40. Michael HaggertyMay 17, 2014
  41. Ronnie SahlbergMay 19, 2014
  42. Michael HaggertyMay 23, 2014
  43. Ronnie SahlbergMay 23, 2014
  44. Michael HaggertyMay 23, 2014
  45. Ronnie SahlbergMay 27, 2014
  46. 26/44 fast-import.c: use a ref transaction when dumping tagsRonnie Sahlberg, May 16, 2014
  47. 27/44 walker.c: use ref transaction for ref updatesRonnie Sahlberg, May 16, 2014
  48. 28/44 refs.c: make write_ref_sha1 staticRonnie Sahlberg, May 16, 2014
  49. 29/44 refs.c: make lock_ref_sha1 staticRonnie Sahlberg, May 16, 2014
  50. 30/44 refs.c: add transaction.status and track OPEN/CLOSED/ERRORRonnie Sahlberg, May 16, 2014
  51. 31/44 refs.c: remove the update_ref_lock functionRonnie Sahlberg, May 16, 2014
  52. 32/44 refs.c: remove the update_ref_write functionRonnie Sahlberg, May 16, 2014
  53. 33/44 refs.c: remove lock_ref_sha1Ronnie Sahlberg, May 16, 2014
  54. 34/44 refs.c: make prune_ref use a transaction to delete the refRonnie Sahlberg, May 16, 2014
  55. 35/44 refs.c: make delete_ref use a transactionRonnie Sahlberg, May 16, 2014
  56. 36/44 refs.c: pass the ref log message to _create/delete/update instead of _commitRonnie Sahlberg, May 16, 2014
  57. 37/44 refs.c: pass NULL as *flags to read_ref_fullRonnie Sahlberg, May 16, 2014
  58. 38/44 refs.c: pack all refs before we start to rename a refRonnie Sahlberg, May 16, 2014
  59. 39/44 refs.c: move the check for valid refname to lock_ref_sha1_basicRonnie Sahlberg, May 16, 2014
  60. 40/44 refs.c: call lock_ref_sha1_basic directly from commitRonnie Sahlberg, May 16, 2014
  61. 41/44 refs.c: add a new flag for transaction delete for refs we know are packed onlyRonnie Sahlberg, May 16, 2014
  62. 42/44 refs.c: pass a skip list to name_conflict_fnRonnie Sahlberg, May 16, 2014
  63. 43/44 refs.c: make rename_ref use a transactionRonnie Sahlberg, May 16, 2014
  64. 44/44 refs.c: remove forward declaration of write_ref_sha1Ronnie Sahlberg, May 16, 2014
  65. Jonathan NiederMay 16, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.