git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v9 27/44] walker.c: use ref transaction for ref updates

From
Ronnie Sahlberg <sahlberg@google.com>
Date
May 15, 2014, 23:15 UTC
Message-ID
<1400195741-22996-28-git-send-email-sahlberg@google.com>
In-Reply-To
<1400195741-22996-1-git-send-email-sahlberg@google.com>

Switch to using ref transactions in walker_fetch(). As part of the refactoring to use ref transactions we also fix a potential memory leak where in the original code if write_ref_sha1() would fail we would end up returning from the function without free()ing the msg string.

This changes the locking slightly for walker_fetch. Previously the code would lock all refs before writing them but now we do not lock the refs until the commit stage. There is thus a very short window where changes could be done locally during the fetch which would be overwritten when the fetch completes and commits its transaction. But this window should be reasonably short. Even if this race does trigger, since both the old code and the new code just overwrites the refs to the new values without checking or comparing them with the previous value, this is not too dissimilar to a similar scenario where you first do a ref change locally and then later do a fetch that overwrites the local change. With this in mind I do not see the change in locking semantics to be critical.

Note that this function is only called when fetching from a remote HTTP repository onto the local (most of the time single-user) repository which likely means that the type of collissions that the previous locking would protect against and cause the fetch to fail for to be even more rare.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 walker.c | 51 ++++++++++++++++++++++++++-------------------------
 1 file changed, 26 insertions(+), 25 deletions(-)
diff --git a/walker.c b/walker.c
index 1dd86b8..cbe5686 100644
--- a/walker.c
+++ b/walker.c
@@ -251,24 +251,18 @@ void walker_targets_free(int targets, char **target, const char **write_ref)
 int walker_fetch(struct walker *walker, int targets, char **target,
 		 const char **write_ref, const char *write_ref_log_details)
 {
-	struct ref_lock **lock = xcalloc(targets, sizeof(struct ref_lock *));
+	char ref_name[PATH_MAX];
+	struct strbuf err = STRBUF_INIT;
+	struct ref_transaction *transaction;
 	unsigned char *sha1 = xmalloc(targets * 20);
-	char *msg;
-	int ret;
+	char *msg = NULL;
 	int i;
 
 	save_commit_buffer = 0;
 
-	for (i = 0; i < targets; i++) {
-		if (!write_ref || !write_ref[i])
-			continue;
-
-		lock[i] = lock_ref_sha1(write_ref[i], NULL);
-		if (!lock[i]) {
-			error("Can't lock ref %s", write_ref[i]);
-			goto unlock_and_fail;
-		}
-	}
+	transaction = ref_transaction_begin();
+	if (!transaction)
+		return -1;
 
 	if (!walker->get_recover)
 		for_each_ref(mark_complete, NULL);
@@ -276,14 +270,14 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 	for (i = 0; i < targets; i++) {
 		if (interpret_target(walker, target[i], &sha1[20 * i])) {
 			error("Could not interpret response from server '%s' as something to pull", target[i]);
-			goto unlock_and_fail;
+			goto rollback_and_fail;
 		}
 		if (process(walker, lookup_unknown_object(&sha1[20 * i])))
-			goto unlock_and_fail;
+			goto rollback_and_fail;
 	}
 
 	if (loop(walker))
-		goto unlock_and_fail;
+		goto rollback_and_fail;
 
 	if (write_ref_log_details) {
 		msg = xmalloc(strlen(write_ref_log_details) + 12);
@@ -294,19 +288,26 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 	for (i = 0; i < targets; i++) {
 		if (!write_ref || !write_ref[i])
 			continue;
-		ret = write_ref_sha1(lock[i], &sha1[20 * i], msg ? msg : "fetch (unknown)");
-		lock[i] = NULL;
-		if (ret)
-			goto unlock_and_fail;
+		sprintf(ref_name, "refs/%s", write_ref[i]);
+		if (ref_transaction_update(transaction, ref_name,
+					   &sha1[20 * i], NULL,
+					   0, 0, &err))
+			goto rollback_and_fail;
+	}
+
+	if (ref_transaction_commit(transaction, msg ? msg : "fetch (unknown)",
+				   &err)) {
+		error("%s", err.buf);
+		goto rollback_and_fail;
 	}
-	free(msg);
 
+	free(msg);
 	return 0;
 
-unlock_and_fail:
-	for (i = 0; i < targets; i++)
-		if (lock[i])
-			unlock_ref(lock[i]);
+rollback_and_fail:
+	free(msg);
+	strbuf_release(&err);
+	ref_transaction_free(transaction);
 
 	return -1;
 }

-- 
2.0.0.rc3.477.gffe78a2
Previous: Ronnie SahlbergNext: Ronnie Sahlberg
Message 28 of 45 in “Use ref transactions for all ref updates”
  1. 00/44 Use ref transactions for all ref updatesRonnie Sahlberg, May 15, 2014
  2. 01/44 refs.c: constify the sha arguments for ref_transaction_create|delete|updateRonnie Sahlberg, May 15, 2014
  3. 02/44 refs.c: allow passing NULL to ref_transaction_freeRonnie Sahlberg, May 15, 2014
  4. 03/44 refs.c: add a strbuf argument to ref_transaction_commit for error loggingRonnie Sahlberg, May 15, 2014
  5. 04/44 refs.c: add an err argument to repack_without_refsRonnie Sahlberg, May 15, 2014
  6. 05/44 refs.c: make ref_update_reject_duplicates take a strbuf argument for errorsRonnie Sahlberg, May 15, 2014
  7. 06/44 refs.c: add an err argument to delete_ref_looseRonnie Sahlberg, May 15, 2014
  8. 07/44 refs.c: make update_ref_write update a strbuf on failureRonnie Sahlberg, May 15, 2014
  9. 08/44 update-ref.c: log transaction error from the update_refRonnie Sahlberg, May 15, 2014
  10. 09/44 refs.c: remove the onerr argument to ref_transaction_commitRonnie Sahlberg, May 15, 2014
  11. 10/44 refs.c: change ref_transaction_update() to do error checking and return statusRonnie Sahlberg, May 15, 2014
  12. 11/44 refs.c: change ref_transaction_create to do error checking and return statusRonnie Sahlberg, May 15, 2014
  13. 12/44 refs.c: ref_transaction_delete to check for error and return statusRonnie Sahlberg, May 15, 2014
  14. 13/44 tag.c: use ref transactions when doing updatesRonnie Sahlberg, May 15, 2014
  15. 14/44 replace.c: use the ref transaction functions for updatesRonnie Sahlberg, May 15, 2014
  16. 15/44 commit.c: use ref transactions for updatesRonnie Sahlberg, May 15, 2014
  17. 16/44 sequencer.c: use ref transactions for all ref updatesRonnie Sahlberg, May 15, 2014
  18. 17/44 fast-import.c: change update_branch to use ref transactionsRonnie Sahlberg, May 15, 2014
  19. 18/44 branch.c: use ref transaction for all ref updatesRonnie Sahlberg, May 15, 2014
  20. 19/44 refs.c: change update_ref to use a transactionRonnie Sahlberg, May 15, 2014
  21. 20/44 refs.c: free the transaction before returning when number of updates is 0Ronnie Sahlberg, May 15, 2014
  22. 21/44 refs.c: ref_transaction_commit should not free the transactionRonnie Sahlberg, May 15, 2014
  23. 22/44 fetch.c: clear errno before calling functions that might set itRonnie Sahlberg, May 15, 2014
  24. 23/44 fetch.c: change s_update_ref to use a ref transactionRonnie Sahlberg, May 15, 2014
  25. 24/44 fetch.c: use a single ref transaction for all ref updatesRonnie Sahlberg, May 15, 2014
  26. 25/44 receive-pack.c: use a reference transaction for updating the refsRonnie Sahlberg, May 15, 2014
  27. 26/44 fast-import.c: use a ref transaction when dumping tagsRonnie Sahlberg, May 15, 2014
  28. 27/44 walker.c: use ref transaction for ref updatesRonnie Sahlberg, May 15, 2014
  29. 28/44 refs.c: make write_ref_sha1 staticRonnie Sahlberg, May 15, 2014
  30. 29/44 refs.c: make lock_ref_sha1 staticRonnie Sahlberg, May 15, 2014
  31. 30/44 refs.c: add transaction.status and track OPEN/CLOSED/ERRORRonnie Sahlberg, May 15, 2014
  32. 31/44 refs.c: remove the update_ref_lock functionRonnie Sahlberg, May 15, 2014
  33. 32/44 refs.c: remove the update_ref_write functionRonnie Sahlberg, May 15, 2014
  34. 33/44 refs.c: remove lock_ref_sha1Ronnie Sahlberg, May 15, 2014
  35. 34/44 refs.c: make prune_ref use a transaction to delete the refRonnie Sahlberg, May 15, 2014
  36. 35/44 refs.c: make delete_ref use a transactionRonnie Sahlberg, May 15, 2014
  37. 36/44 refs.c: pass the ref log message to _create/delete/update instead of _commitRonnie Sahlberg, May 15, 2014
  38. 37/44 refs.c: pass NULL as *flags to read_ref_fullRonnie Sahlberg, May 15, 2014
  39. 38/44 refs.c: pack all refs before we start to rename a refRonnie Sahlberg, May 15, 2014
  40. 39/44 refs.c: move the check for valid refname to lock_ref_sha1_basicRonnie Sahlberg, May 15, 2014
  41. 40/44 refs.c: call lock_ref_sha1_basic directly from commitRonnie Sahlberg, May 15, 2014
  42. 41/44 refs.c: add a new flag for transaction delete for refs we know are packed onlyRonnie Sahlberg, May 15, 2014
  43. 42/44 refs.c: pass a skip list to name_conflict_fnRonnie Sahlberg, May 15, 2014
  44. 43/44 refs.c: make rename_ref use a transactionRonnie Sahlberg, May 15, 2014
  45. 44/44 refs.c: remove forward declaration of write_ref_sha1Ronnie Sahlberg, May 15, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.