git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 4/8] refs.c: add transaction function to append to the reflog

From
Stefan Beller <sbeller@google.com>
Date
Dec 6, 2014, 02:46 UTC
Message-ID
<1417833995-25687-5-git-send-email-sbeller@google.com>
In-Reply-To
<1417833995-25687-1-git-send-email-sbeller@google.com>

Currently a transaction can include one or more reflog updates. But these are not part of the contract of the transaction committing all or nothing.

Introduce a function transaction_update_reflog, which adds a reflog update to a transaction. Later patches will update code that writes to reflogs to use this function.

This will allow us to write code such as:
	t = transaction_begin()
	transaction_truncate_reflog(t, "foo"); // introduced by next patch
	loop-over-something...
		if (want_reflog_entry(...))
			transaction_reflog_update(t, "foo", 0, <message>);
	transaction_commit(t)

transaction_ref_update still updates the reflog along with its ref update. A later patch will make it use transaction_update_reflog internally.

Unlike transaction_update_ref, this writes out the proposed contents of the reflog to a temporary file at transaction_reflog_update time instead of waiting for the transaction waiting to be committed. This avoids an explosion of memory usage when writing lots of reflog updates within a single transaction.

This requires changing where the temporary file is located. If the temporary file is located at $GIT_DIR/logs/$REF_NAME.lock, then a transaction that tries to remove refs/heads/foo and introduce refs/heads/foo/bar would run into a directory/file conflict when trying to write to $GIT_DIR/logs/refs/heads/foo/bar.lock because the reflog for branch foo is not safe to remove yet.

We work around this by placing the temporary files at $GIT_DIR/logs/lock/$REF_NAME.lock Putting the temporary file under .git/logs ensures it's likely that it will be in the same file system as the reflogs and can be atomically renamed into place.

During one transaction we allow to make multiple reflog updates to the same ref. This means we only need to lock the reflog once during the first update that touches the reflog and that all further updates can just write the reflog entry since the reflog is already locked.

transaction_update_reflog should lock the corresponding ref to avoid having to compete with other reflog updates, but this will be deferred to a later patch for simplicity.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
Notes:
    This has been sent to the list before, but now it is split up
    into this and the next patch.
    The commit message was overhauled, a huge thanks to Jonathan!
    
    There are no major changes in the code.
 refs.c | 94 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 92 insertions(+), 2 deletions(-)
diff --git a/refs.c b/refs.c
index aad44d5..d767418 100644
--- a/refs.c
+++ b/refs.c
@@ -3559,19 +3559,30 @@ struct transaction {
 	struct ref_update **ref_updates;
 	size_t alloc;
 	size_t nr;
+
+	/*
+	 * Sorted list of reflogs to be committed.
+	 * the util points to the lock_file.
+	 */
+	struct string_list reflog_updates;
 	enum transaction_state state;
 };
 
 struct transaction *transaction_begin(struct strbuf *err)
 {
+	struct transaction *ret;
+
 	assert(err);
 
-	return xcalloc(1, sizeof(struct transaction));
+	ret = xcalloc(1, sizeof(struct transaction));
+	ret->reflog_updates.strdup_strings = 1;
+	return ret;
 }
 
 void transaction_free(struct transaction *transaction)
 {
 	int i;
+	struct string_list_item *item;
 
 	if (!transaction)
 		return;
@@ -3581,6 +3592,12 @@ void transaction_free(struct transaction *transaction)
 		free(transaction->ref_updates[i]);
 	}
 	free(transaction->ref_updates);
+
+	for_each_string_list_item(item, &transaction->reflog_updates) {
+		rollback_lock_file(item->util);
+	}
+	string_list_clear(&transaction->reflog_updates, 0);
+
 	free(transaction);
 }
 
@@ -3631,6 +3648,71 @@ int transaction_update_ref(struct transaction *transaction,
 	return 0;
 }
 
+/*
+ * Append a reflog entry for refname.
+ */
+static int transaction_update_reflog(struct transaction *transaction,
+				     const char *refname,
+				     const unsigned char *new_sha1,
+				     const unsigned char *old_sha1,
+				     const char *email,
+				     unsigned long timestamp, int tz,
+				     const char *msg,
+				     struct strbuf *err)
+{
+	struct lock_file *lock;
+	struct strbuf buf = STRBUF_INIT;
+	struct string_list_item *item;
+
+	if (transaction->state != TRANSACTION_OPEN)
+		die("BUG: update_reflog called for transaction that is not open");
+
+	item = string_list_insert(&transaction->reflog_updates, refname);
+	if (!item->util) {
+		int infd;
+		char *path = git_path("logs/locks/%s", refname);
+		lock = xcalloc(1, sizeof(struct lock_file));
+		item->util = lock;
+		if (safe_create_leading_directories(path)) {
+			strbuf_addf(err, "could not create leading directories of '%s': %s",
+				    path, strerror(errno));
+			goto failure;
+		}
+
+		if (hold_lock_file_for_update(lock, path, 0) < 0) {
+			unable_to_lock_message(path, errno, err);
+			goto failure;
+		}
+
+		/* copy existing reflog over */
+		if ((infd = open(git_path("logs/%s", refname), O_RDONLY)) < 0) {
+			strbuf_addf(err, "could not open the reflog '%s': %s",
+				    path, strerror(errno));
+			goto failure;
+		}
+		if (copy_fd(infd, lock->fd))
+			goto failure;
+		close(infd);
+	}
+	lock = item->util;
+
+	if (email)
+		strbuf_addf(&buf, "%s %lu %+05d", email, timestamp, tz);
+
+	if (log_ref_write_fd(lock->fd, old_sha1, new_sha1, buf.buf, msg)) {
+		strbuf_addf(err, "Could not write to reflog: '%s': %s",
+			    refname, strerror(errno));
+		goto failure;
+	}
+
+	strbuf_release(&buf);
+	return 0;
+failure:
+	strbuf_release(&buf);
+	transaction->state = TRANSACTION_CLOSED;
+	return -1;
+}
+
 int transaction_create_ref(struct transaction *transaction,
 			   const char *refname,
 			   const unsigned char *new_sha1,
@@ -3715,13 +3797,14 @@ int transaction_commit(struct transaction *transaction,
 	const char **delnames;
 	int n = transaction->nr;
 	struct ref_update **updates = transaction->ref_updates;
+	struct string_list_item *item;
 
 	assert(err);
 
 	if (transaction->state != TRANSACTION_OPEN)
 		die("BUG: commit called for transaction that is not open");
 
-	if (!n) {
+	if (!n && !transaction->reflog_updates.nr) {
 		transaction->state = TRANSACTION_CLOSED;
 		return 0;
 	}
@@ -3798,6 +3881,13 @@ int transaction_commit(struct transaction *transaction,
 	}
 	for (i = 0; i < delnum; i++)
 		unlink_or_warn(git_path("logs/%s", delnames[i]));
+
+	/* Commit all reflog updates*/
+	for_each_string_list_item(item, &transaction->reflog_updates) {
+		struct lock_file *lock = item->util;
+		commit_lock_file_to(lock, git_path("logs/%s", item->string));
+	}
+
 	clear_loose_ref_cache(&ref_cache);
 
 cleanup:

-- 
2.2.0
Previous: Stefan BellerNext: Junio C Hamano
Message 7 of 18 in “Making reflog modifications part of the transactions API”
  1. 0/8 Making reflog modifications part of the transactions APIStefan Beller, Dec 6, 2014
  2. 1/8 refs.c: let fprintf handle the formattingStefan Beller, Dec 6, 2014
  3. 2/8 refs.c: rename the transaction functionsStefan Beller, Dec 6, 2014
  4. Junio C HamanoDec 11, 2014
  5. Stefan BellerDec 11, 2014
  6. 3/8 refs.c: rename transaction.updates to transaction.ref_updatesStefan Beller, Dec 6, 2014
  7. 4/8 refs.c: add transaction function to append to the reflogStefan Beller, Dec 6, 2014
  8. Junio C HamanoDec 11, 2014
  9. 5/8 refs.c: add transaction function to delete the reflogStefan Beller, Dec 6, 2014
  10. 6/8 refs.c: use a reflog transaction when writing during expireStefan Beller, Dec 6, 2014
  11. 7/8 refs.c: rename log_ref_setup to create_reflogStefan Beller, Dec 6, 2014
  12. 8/8 refs.c: allow deleting refs with a broken sha1Stefan Beller, Dec 6, 2014
  13. Stefan BellerDec 8, 2014
  14. Jonathan NiederDec 8, 2014
  15. Michael HaggertyDec 12, 2014
  16. Stefan BellerDec 12, 2014
  17. ronnie sahlbergDec 12, 2014
  18. Michael HaggertyDec 14, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.