git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 12/23] ref_transaction_commit(): break into multiple functions

From
Michael Haggerty <mhagger@alum.mit.edu>
Date
May 17, 2017, 12:05 UTC
Message-ID
<5ec5f3f9b4c9fc9f719d116b48dcf7b61e74ae49.1495014840.git.mhagger@alum.mit.edu>
In-Reply-To
<cover.1495014840.git.mhagger@alum.mit.edu>

Break the function `ref_transaction_commit()` into two functions, `ref_transaction_prepare()` and `ref_transaction_finish()`, with a third function, `ref_transaction_abort()`, that can be used to abort the transaction. Break up the `ref_store` methods similarly.

This split will make it easier to implement compound reference stores, where a transaction might have to span multiple underlying stores. In that case, we would first want to "prepare" the sub-transactions in each of the reference stores. If any of the "prepare" steps fails, we would "abort" all of the sub-transactions. Only if all of the "prepare" steps succeed would we "finish" each of them.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 refs.c               | 34 ++++++++++++++++++++++---
 refs.h               | 37 ++++++++++++++++++++++++++-
 refs/files-backend.c | 71 +++++++++++++++++++++++++++++++++++++++++-----------
 refs/refs-internal.h | 42 ++++++++++++++++++++++++-------
 4 files changed, 157 insertions(+), 27 deletions(-)
diff --git a/refs.c b/refs.c
index 14dec88e7f..689362db1e 100644
--- a/refs.c
+++ b/refs.c
@@ -1689,8 +1689,8 @@ int create_symref(const char *ref_target, const char *refs_heads_master,
 				  refs_heads_master, logmsg);
 }
 
-int ref_transaction_commit(struct ref_transaction *transaction,
-			   struct strbuf *err)
+int ref_transaction_prepare(struct ref_transaction *transaction,
+			    struct strbuf *err)
 {
 	struct ref_store *refs = transaction->ref_store;
 
@@ -1700,7 +1700,35 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 		return -1;
 	}
 
-	return refs->be->transaction_commit(refs, transaction, err);
+	return refs->be->transaction_prepare(refs, transaction, err);
+}
+
+int ref_transaction_finish(struct ref_transaction *transaction,
+			   struct strbuf *err)
+{
+	struct ref_store *refs = transaction->ref_store;
+
+	return refs->be->transaction_finish(refs, transaction, err);
+}
+
+int ref_transaction_abort(struct ref_transaction *transaction,
+			  struct strbuf *err)
+{
+	struct ref_store *refs = transaction->ref_store;
+
+	return refs->be->transaction_abort(refs, transaction, err);
+}
+
+int ref_transaction_commit(struct ref_transaction *transaction,
+			   struct strbuf *err)
+{
+	int ret;
+
+	ret = ref_transaction_prepare(transaction, err);
+	if (ret)
+		return ret;
+
+	return ref_transaction_finish(transaction, err);
 }
 
 int refs_verify_refname_available(struct ref_store *refs,
diff --git a/refs.h b/refs.h
index b2d9626be6..4139e917f5 100644
--- a/refs.h
+++ b/refs.h
@@ -525,7 +525,10 @@ int ref_transaction_verify(struct ref_transaction *transaction,
 
 /*
  * Commit all of the changes that have been queued in transaction, as
- * atomically as possible.
+ * atomically as possible. This is equivalent to calling
+ * `ref_transaction_prepare()` then `ref_transaction_finish()` (see
+ * below), which you can call yourself if you want finer control over
+ * reference updates.
  *
  * Returns 0 for success, or one of the below error codes for errors.
  */
@@ -536,6 +539,38 @@ int ref_transaction_verify(struct ref_transaction *transaction,
 int ref_transaction_commit(struct ref_transaction *transaction,
 			   struct strbuf *err);
 
+/*
+ * Perform the preparatory stages of commiting `transaction`. Acquire
+ * any needed locks, check preconditions, etc.; basically, do as much
+ * as possible to ensure that the transaction will be able to go
+ * through, stopping just short of making any irrevocable or
+ * user-visible changes. Anything that this function does should be
+ * able to be finished up by calling `ref_transaction_finish()` or
+ * rolled back by calling `ref_transaction_abort()`.
+ *
+ * On success, return 0 and leave the transaction in "prepared" state.
+ * On failure, abort the transaction and return one of the
+ * `TRANSACTION_*` constants.
+ *
+ * Callers who don't need such fine-grained control over commiting
+ * reference transactions should just call `ref_transaction_commit()`.
+ */
+int ref_transaction_prepare(struct ref_transaction *transaction,
+			    struct strbuf *err);
+
+/*
+ * Perform the final commit of `transaction`, which has already been
+ * prepared.
+ */
+int ref_transaction_finish(struct ref_transaction *transaction,
+			   struct strbuf *err);
+
+/*
+ * Abort `transaction`, which has already been prepared.
+ */
+int ref_transaction_abort(struct ref_transaction *transaction,
+			  struct strbuf *err);
+
 /*
  * Like ref_transaction_commit(), but optimized for creating
  * references when originally initializing a repository (e.g., by "git
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 7ddd4f87d5..f48409132d 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2867,27 +2867,24 @@ static void files_transaction_cleanup(struct ref_transaction *transaction)
 	transaction->state = REF_TRANSACTION_CLOSED;
 }
 
-static int files_transaction_commit(struct ref_store *ref_store,
-				    struct ref_transaction *transaction,
-				    struct strbuf *err)
+static int files_transaction_prepare(struct ref_store *ref_store,
+				     struct ref_transaction *transaction,
+				     struct strbuf *err)
 {
 	struct files_ref_store *refs =
 		files_downcast(ref_store, REF_STORE_WRITE,
-			       "ref_transaction_commit");
+			       "ref_transaction_prepare");
 	size_t i;
 	int ret = 0;
-	struct string_list refs_to_delete = STRING_LIST_INIT_NODUP;
-	struct string_list_item *ref_to_delete;
 	struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
 	char *head_ref = NULL;
 	int head_type;
 	struct object_id head_oid;
-	struct strbuf sb = STRBUF_INIT;
 
 	assert(err);
 
 	if (transaction->state != REF_TRANSACTION_OPEN)
-		die("BUG: commit called for transaction that is not open");
+		die("BUG: prepare called for transaction that is not open");
 
 	if (!transaction->nr)
 		goto cleanup;
@@ -2949,6 +2946,8 @@ static int files_transaction_commit(struct ref_store *ref_store,
 	 * that new values are valid, and write new values to the
 	 * lockfiles, ready to be activated. Only keep one lockfile
 	 * open at a time to avoid running out of file descriptors.
+	 * Note that lock_ref_for_update() might append more updates
+	 * to the transaction.
 	 */
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
@@ -2956,7 +2955,41 @@ static int files_transaction_commit(struct ref_store *ref_store,
 		ret = lock_ref_for_update(refs, update, transaction,
 					  head_ref, &affected_refnames, err);
 		if (ret)
-			goto cleanup;
+			break;
+	}
+
+cleanup:
+	free(head_ref);
+	string_list_clear(&affected_refnames, 0);
+
+	if (ret)
+		files_transaction_cleanup(transaction);
+	else
+		transaction->state = REF_TRANSACTION_PREPARED;
+
+	return ret;
+}
+
+static int files_transaction_finish(struct ref_store *ref_store,
+				    struct ref_transaction *transaction,
+				    struct strbuf *err)
+{
+	struct files_ref_store *refs =
+		files_downcast(ref_store, 0, "ref_transaction_finish");
+	size_t i;
+	int ret = 0;
+	struct string_list refs_to_delete = STRING_LIST_INIT_NODUP;
+	struct string_list_item *ref_to_delete;
+	struct strbuf sb = STRBUF_INIT;
+
+	assert(err);
+
+	if (transaction->state != REF_TRANSACTION_PREPARED)
+		die("BUG: finish called for transaction that is not prepared");
+
+	if (!transaction->nr) {
+		transaction->state = REF_TRANSACTION_CLOSED;
+		return 0;
 	}
 
 	/* Perform updates first so live commits remain referenced */
@@ -3037,7 +3070,6 @@ static int files_transaction_commit(struct ref_store *ref_store,
 
 cleanup:
 	files_transaction_cleanup(transaction);
-	strbuf_release(&sb);
 
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
@@ -3054,13 +3086,22 @@ static int files_transaction_commit(struct ref_store *ref_store,
 		}
 	}
 
+	strbuf_release(&sb);
 	string_list_clear(&refs_to_delete, 0);
-	free(head_ref);
-	string_list_clear(&affected_refnames, 0);
-
 	return ret;
 }
 
+static int files_transaction_abort(struct ref_store *ref_store,
+				   struct ref_transaction *transaction,
+				   struct strbuf *err)
+{
+	if (transaction->state != REF_TRANSACTION_PREPARED)
+		die("BUG: abort called for transaction that is not prepared");
+
+	files_transaction_cleanup(transaction);
+	return 0;
+}
+
 static int ref_present(const char *refname,
 		       const struct object_id *oid, int flags, void *cb_data)
 {
@@ -3327,7 +3368,9 @@ struct ref_storage_be refs_be_files = {
 	"files",
 	files_ref_store_create,
 	files_init_db,
-	files_transaction_commit,
+	files_transaction_prepare,
+	files_transaction_finish,
+	files_transaction_abort,
 	files_initial_transaction_commit,
 
 	files_pack_refs,
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 9ed4387c8c..2ff65d3ebd 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -185,17 +185,26 @@ struct ref_update *ref_transaction_add_update(
 
 /*
  * Transaction states.
- * OPEN:   The transaction is in a valid state and can accept new updates.
- *         An OPEN transaction can be committed.
- * CLOSED: A closed transaction is no longer active and no other operations
- *         than free can be used on it in this state.
- *         A transaction can either become closed by successfully committing
- *         an active transaction or if there is a failure while building
- *         the transaction thus rendering it failed/inactive.
+ *
+ * OPEN:   The transaction is in a valid state and new updates can be
+ *         added to it. An OPEN transaction can be prepared or
+ *         committed.
+ *
+ * PREPARED: ref_transaction_prepare(), which locks all of the
+ *         references involved in the update and checks that the
+ *         update has no errors, has been called successfully for the
+ *         transaction.
+ *
+ * CLOSED: The transaction is no longer active. No other operations
+ *         than free can be used on it in this state. A transaction
+ *         becomes closed if there is a failure while building the
+ *         transaction, if an active transaction is committed, or if a
+ *         prepared transaction is finished.
  */
 enum ref_transaction_state {
 	REF_TRANSACTION_OPEN   = 0,
-	REF_TRANSACTION_CLOSED = 1
+	REF_TRANSACTION_PREPARED = 1,
+	REF_TRANSACTION_CLOSED = 2
 };
 
 /*
@@ -497,6 +506,18 @@ typedef struct ref_store *ref_store_init_fn(const char *gitdir,
 
 typedef int ref_init_db_fn(struct ref_store *refs, struct strbuf *err);
 
+typedef int ref_transaction_prepare_fn(struct ref_store *refs,
+				       struct ref_transaction *transaction,
+				       struct strbuf *err);
+
+typedef int ref_transaction_finish_fn(struct ref_store *refs,
+				      struct ref_transaction *transaction,
+				      struct strbuf *err);
+
+typedef int ref_transaction_abort_fn(struct ref_store *refs,
+				     struct ref_transaction *transaction,
+				     struct strbuf *err);
+
 typedef int ref_transaction_commit_fn(struct ref_store *refs,
 				      struct ref_transaction *transaction,
 				      struct strbuf *err);
@@ -599,7 +620,10 @@ struct ref_storage_be {
 	const char *name;
 	ref_store_init_fn *init;
 	ref_init_db_fn *init_db;
-	ref_transaction_commit_fn *transaction_commit;
+
+	ref_transaction_prepare_fn *transaction_prepare;
+	ref_transaction_finish_fn *transaction_finish;
+	ref_transaction_abort_fn *transaction_abort;
 	ref_transaction_commit_fn *initial_transaction_commit;
 
 	pack_refs_fn *pack_refs;

-- 
2.11.0
Previous: Brandon WilliamsNext: Stefan Beller
Message 39 of 73 in “Prepare to separate out a packed_ref_store”
  1. 00/23 Prepare to separate out a packed_ref_storeMichael Haggerty, May 17, 2017
  2. 01/23 t3600: clean up permissions test properlyMichael Haggerty, May 17, 2017
  3. Jeff KingMay 17, 2017
  4. Michael HaggertyMay 17, 2017
  5. Junio C HamanoMay 18, 2017
  6. Michael HaggertyMay 19, 2017
  7. 02/23 refs.h: clarify docstring for the ref_transaction_update()-related fnsMichael Haggerty, May 17, 2017
  8. Stefan BellerMay 17, 2017
  9. Junio C HamanoMay 18, 2017
  10. 03/23 ref_iterator_begin_fn(): fix docstringMichael Haggerty, May 17, 2017
  11. 05/23 refs_ref_iterator_begin(): don't check prefixes redundantlyMichael Haggerty, May 17, 2017
  12. Jeff KingMay 17, 2017
  13. Michael HaggertyMay 17, 2017
  14. 07/23 ref_store: take `logmsg` parameter when deleting referencesMichael Haggerty, May 17, 2017
  15. Jeff KingMay 17, 2017
  16. Michael HaggertyMay 17, 2017
  17. Jeff KingMay 17, 2017
  18. 04/23 prefix_ref_iterator: don't trim too muchMichael Haggerty, May 17, 2017
  19. Jeff KingMay 17, 2017
  20. Michael HaggertyMay 17, 2017
  21. Jeff KingMay 17, 2017
  22. Junio C HamanoMay 18, 2017
  23. Michael HaggertyMay 18, 2017
  24. 09/23 files-backend: move `lock` member to `files_ref_store`Michael Haggerty, May 17, 2017
  25. Jeff KingMay 17, 2017
  26. Michael HaggertyMay 17, 2017
  27. 06/23 refs: use `size_t` indexes when iterating over ref transaction updatesMichael Haggerty, May 17, 2017
  28. Stefan BellerMay 17, 2017
  29. Michael HaggertyMay 18, 2017
  30. 10/23 files_ref_store: put the packed files lock directly in this structMichael Haggerty, May 17, 2017
  31. Jeff KingMay 17, 2017
  32. Michael HaggertyMay 17, 2017
  33. Stefan BellerMay 17, 2017
  34. Brandon WilliamsMay 18, 2017
  35. Michael HaggertyMay 19, 2017
  36. Brandon WilliamsMay 18, 2017
  37. Jeff KingMay 18, 2017
  38. Brandon WilliamsMay 18, 2017
  39. 12/23 ref_transaction_commit(): break into multiple functionsMichael Haggerty, May 17, 2017
  40. Stefan BellerMay 17, 2017
  41. Michael HaggertyMay 19, 2017
  42. 16/23 should_pack_ref(): new function, extracted from `files_pack_refs()`Michael Haggerty, May 17, 2017
  43. 17/23 get_packed_ref_cache(): assume "packed-refs" won't change while lockedMichael Haggerty, May 17, 2017
  44. Stefan BellerMay 17, 2017
  45. Jeff KingMay 18, 2017
  46. Stefan BellerMay 18, 2017
  47. 18/23 read_packed_refs(): do more of the work of reading packed refsMichael Haggerty, May 17, 2017
  48. 19/23 read_packed_refs(): report unexpected fopen() failuresMichael Haggerty, May 17, 2017
  49. Jeff KingMay 17, 2017
  50. Michael HaggertyMay 17, 2017
  51. Junio C HamanoMay 18, 2017
  52. Jeff KingMay 18, 2017
  53. 15/23 ref_update_reject_duplicates(): add a sanity checkMichael Haggerty, May 17, 2017
  54. 20/23 refs_ref_iterator_begin(): handle `GIT_REF_PARANOIA`Michael Haggerty, May 17, 2017
  55. Jeff KingMay 17, 2017
  56. 22/23 ref-filter: limit traversal to prefixMichael Haggerty, May 17, 2017
  57. Jeff KingMay 17, 2017
  58. Michael HaggertyMay 19, 2017
  59. 14/23 ref_update_reject_duplicates(): use `size_t` rather than `int`Michael Haggerty, May 17, 2017
  60. 23/23 cache_ref_iterator_begin(): avoid priming unneeded directoriesMichael Haggerty, May 17, 2017
  61. 21/23 create_ref_entry(): remove `check_name` optionMichael Haggerty, May 17, 2017
  62. 08/23 lockfile: add a new method, is_lock_file_locked()Michael Haggerty, May 17, 2017
  63. Jeff KingMay 17, 2017
  64. 11/23 files_transaction_cleanup(): new helper functionMichael Haggerty, May 17, 2017
  65. Jeff KingMay 17, 2017
  66. Michael HaggertyMay 19, 2017
  67. Stefan BellerMay 17, 2017
  68. Michael HaggertyMay 19, 2017
  69. 13/23 ref_update_reject_duplicates(): expose function to whole refs moduleMichael Haggerty, May 17, 2017
  70. Jeff KingMay 17, 2017
  71. Stefan BellerMay 17, 2017
  72. Johannes SixtMay 18, 2017
  73. Jeff KingMay 18, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.