git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 09/31] refs.c: allow multiple reflog updates during a single transaction

From
Ronnie Sahlberg <sahlberg@google.com>
Date
May 19, 2014, 23:06 UTC
Message-ID
<CAL=YDWn5GXR8ADt8a0dJBS-t62xrkuMz_0h=HTqLkpCawoV=xw@mail.gmail.com>
In-Reply-To
<xmqqppjd8l13.fsf@gitster.dls.corp.google.com>
On Fri, May 16, 2014 at 2:35 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 58 quoted lines
> Ronnie Sahlberg <sahlberg@google.com> writes:
>
>> Allow to make multiple reflog updates to the same ref during a transaction.
>> This means we only need to lock the reflog once, during the first update that
>> touches the reflog, and that all further updates can just write the reflog
>> entry since the reflog is already locked.
>>
>> This allows us to write code such as:
>>
>> t = transaction_begin()
>> transaction_reflog_update(t, "foo", REFLOG_TRUNCATE, NULL);
>> loop-over-somehting...
>>    transaction_reflog_update(t, "foo", 0, <message>);
>> transaction_commit(t)
>
> OK, so you are now doing not just "refs" but also "reflogs", you
> felt that "ref_transaction()" does not cover the latter.  Is that
> the reason for the rename in the earlier step?
>
> I am sort-of on the fence.
>
> Calling the begin "ref_transaction_begin" and then calling the new
> function "ref_transaction_log_update" would still allow us to
> differentiate transactions on refs and reflogs, while allowing other
> kinds of transactions that are not related to refs at all to employ
> a mechanism that is different from the one that is used to implement
> the transactions on refs and reflogs you are building here.
>
> But I think I am OK with the generic "transaction-begin" now.
> Having one mechanism for refs and reflogs, and then having another
> completely different mechanism for other things, will not let us
> coordinate between the two easily, so "allow transactions that are
> not related to refs at all to be built on a different mechanism" may
> not be a worthwhile goal to pursue in the first place.  Please
> consider the question on the naming in the earlier one dropped.
>
>>
>> where we first truncate the reflog and then build the new content one line at a
>> time.
>>
>> Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
>> ---
>>  refs.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++++---------
>>  1 file changed, 49 insertions(+), 9 deletions(-)
>>
>> diff --git a/refs.c b/refs.c
>> index a3f60ad..e7ede03 100644
>> --- a/refs.c
>> +++ b/refs.c
>> @@ -37,6 +37,10 @@ static inline int bad_ref_char(int ch)
>>   *  need to lock the loose ref during the transaction.
>>   */
>>  #define REF_ISPACKONLY       0x0200
>> +/** Only the first reflog update needs to lock the reflog file. Further updates
>> + *  just use the lock taken by the first update.
>> + */
>
> Style.
Done.
Show 25 quoted lines
>
>> @@ -3349,8 +3355,23 @@ int transaction_update_reflog(struct ref_transaction *transaction,
>>                             int flags)
>>  {
>>       struct ref_update *update;
>> +     int i;
>>
>>       update = add_update(transaction, refname, UPDATE_LOG);
>> +     update->flags = flags;
>> +     for (i = 0; i < transaction->nr - 1; i++) {
>> +             if (transaction->updates[i]->update_type != UPDATE_LOG)
>> +                     continue;
>> +             if (!strcmp(transaction->updates[i]->refname,
>> +                         update->refname)) {
>> +                     update->flags |= UPDATE_REFLOG_NOLOCK;
>> +                     update->orig_update = transaction->updates[i];
>> +                     break;
>> +             }
>> +     }
>> +     if (!(update->flags & UPDATE_REFLOG_NOLOCK))
>> +       update->reflog_lock = xcalloc(1, sizeof(struct lock_file));
>
> So with two calls to transaction-update-reflog, we make two calls to
> add-update, and each holds a separate lock?  If we write two entries
> to record two updates of the same ref, would we still want to do so?

If we call transaction_update_reflog twice we will call add_update twice since we are doing two updates. But if the two calls are for the same ref then we only lock the reflog for the first update and then in the second update we re-use the lock structure from the first.

Later updates to reflog.c will show how to loop over the old reflog, using the existing iterator, and then create a new expired reflog one entry at a time.

Show 15 quoted lines
>
>> +     /* Rollback any reflog files that are still open */
>> +     for (i = 0; i < n; i++) {
>> +             struct ref_update *update = updates[i];
>> +
>> +             if (update->update_type != UPDATE_LOG)
>> +                     continue;
>> +             if (update->flags & UPDATE_REFLOG_NOLOCK)
>> +                     continue;
>> +             if (update->reflog_fd == -1)
>> +                     continue;
>> +             rollback_lock_file(update->reflog_lock);
>> +     }
>>       transaction->status = ret ? REF_TRANSACTION_ERROR
>>         : REF_TRANSACTION_CLOSED;
Previous: Ronnie SahlbergNext: Ronnie Sahlberg
Message 21 of 43 in “Finish implementing ref and reflog transactions”
  1. 00/31 Finish implementing ref and reflog transactionsRonnie Sahlberg, May 14, 2014
  2. 01/31 refs.c make ref_transaction_create a wrapper to ref_transaction_updateRonnie Sahlberg, May 14, 2014
  3. 02/31 refs.c: make ref_transaction_delete a wrapper for ref_transaction_updateRonnie Sahlberg, May 14, 2014
  4. 03/31 refs.c: rename the transaction functionsRonnie Sahlberg, May 14, 2014
  5. Junio C HamanoMay 16, 2014
  6. Ronnie SahlbergMay 19, 2014
  7. Junio C HamanoMay 19, 2014
  8. 04/31 refs.c: add a new update_type field to ref_updateRonnie Sahlberg, May 14, 2014
  9. 05/31 refs.c: add a function to append a reflog entry to a fdRonnie Sahlberg, May 14, 2014
  10. 06/31 refs.c: add a transaction function to append a reflog entryRonnie Sahlberg, May 14, 2014
  11. 07/31 refs.c: add a flag to allow reflog updates to truncate the logRonnie Sahlberg, May 14, 2014
  12. Junio C HamanoMay 16, 2014
  13. Ronnie SahlbergMay 19, 2014
  14. 08/31 refs.c: only write reflog update if msg is non-NULLRonnie Sahlberg, May 14, 2014
  15. Junio C HamanoMay 16, 2014
  16. Ronnie SahlbergMay 19, 2014
  17. 09/31 refs.c: allow multiple reflog updates during a single transactionRonnie Sahlberg, May 14, 2014
  18. Junio C HamanoMay 16, 2014
  19. Eric SunshineMay 16, 2014
  20. Ronnie SahlbergMay 19, 2014
  21. Ronnie SahlbergMay 19, 2014
  22. 10/31 reflog.c: use a reflog transaction when writing during expireRonnie Sahlberg, May 14, 2014
  23. 11/31 refs.c: null terminate the string in copy_msgRonnie Sahlberg, May 14, 2014
  24. 12/31 refs.c: track the refnames we are deleting in the transaction structureRonnie Sahlberg, May 14, 2014
  25. 13/31 refs.c: update the list of deleted refs during _update instead of _commitRonnie Sahlberg, May 14, 2014
  26. 14/31 refs.c: return error instead of dying when locking fails during transactionRonnie Sahlberg, May 14, 2014
  27. 15/31 refs.c: lock the ref during _update instead of during _commitRonnie Sahlberg, May 14, 2014
  28. 16/31 refs.c: add an error argument to create/delete/update just like commitRonnie Sahlberg, May 14, 2014
  29. 17/31 refs.c: make _update_reflog take an error argumentRonnie Sahlberg, May 14, 2014
  30. 18/31 refs.c: return immediately from _commit if the transaction has an errorRonnie Sahlberg, May 14, 2014
  31. 19/31 tests: move tests for -z update/delete/verify to after the ref is createdRonnie Sahlberg, May 14, 2014
  32. 20/31 refs.c: check for lock conflicts already in _updateRonnie Sahlberg, May 14, 2014
  33. 21/31 refs.c allow multiple updates of the same ref in a transactionRonnie Sahlberg, May 14, 2014
  34. 22/31 refs.c: release all remaining locks during transaction_freeRonnie Sahlberg, May 14, 2014
  35. 23/31 reflog.c: use the existing transaction to also lock and update the refRonnie Sahlberg, May 14, 2014
  36. 24/31 refs.c: make unlock_ref staticRonnie Sahlberg, May 14, 2014
  37. 25/31 refs.c: make close_ref staticRonnie Sahlberg, May 14, 2014
  38. 26/31 refs.c: make commit_ref staticRonnie Sahlberg, May 14, 2014
  39. 27/31 refs.c: remove the function lock_any_ref_for_updateRonnie Sahlberg, May 14, 2014
  40. 28/31 refs.c: make struct ref_lock private to refs.cRonnie Sahlberg, May 14, 2014
  41. 29/31 refs.c: allow passing raw git_committer_info as email to _update_reflogRonnie Sahlberg, May 14, 2014
  42. 30/31 refs.c: move ref_update and other definitions to earlier in the fileRonnie Sahlberg, May 14, 2014
  43. 31/31 refs.c: use the transaction to manage the reflog in rename_refsRonnie Sahlberg, May 14, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.