git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 04/10] refs/files: support symbolic and root refs in initial transaction

From
Patrick Steinhardt <ps@pks.im>
Date
Nov 25, 2024, 06:27 UTC
Message-ID
<20241125-pks-refs-optimize-migrations-v3-4-17bc85e33ad7@pks.im>
In-Reply-To
<20241125-pks-refs-optimize-migrations-v3-0-17bc85e33ad7@pks.im>

The "files" backend has implemented special logic when committing the first transactions in an otherwise empty ref store: instead of writing all refs as separate loose files, it instead knows to write them all into a "packed-refs" file directly. This is significantly more efficient than having to write each of the refs as separate "loose" ref.

The only user of this optimization is git-clone(1), which only uses this mechanism to write regular refs. Consequently, the implementation does not know how to handle both symbolic and root refs. While fine in the context of git-clone(1), this keeps us from using the mechanism in more cases.

Adapt the logic to also support symbolic and root refs by using a second transaction that we use for all of the refs that need to be written as loose refs.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 refs/files-backend.c | 44 ++++++++++++++++++++++++++++++++++----------
 1 file changed, 34 insertions(+), 10 deletions(-)
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 3ed18475a72aa4798d15b2912c37b4caabd47aac..116d4259697b20583cb2db34ed47025e8781cd42 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2995,6 +2995,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
 	int ret = 0;
 	struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
 	struct ref_transaction *packed_transaction = NULL;
+	struct ref_transaction *loose_transaction = NULL;
 
 	assert(err);
 
@@ -3040,6 +3041,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
 		if ((update->flags & REF_HAVE_OLD) &&
 		    !is_null_oid(&update->old_oid))
 			BUG("initial ref transaction with old_sha1 set");
+
 		if (refs_verify_refname_available(&refs->base, update->refname,
 						  &affected_refnames, NULL,
 						  err)) {
@@ -3048,26 +3050,48 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
 		}
 
 		/*
-		 * Add a reference creation for this reference to the
-		 * packed-refs transaction:
+		 * packed-refs don't support symbolic refs and root refs, so we
+		 * have to queue these references via the loose transaction.
 		 */
-		ref_transaction_add_update(packed_transaction, update->refname,
-					   update->flags & ~REF_HAVE_OLD,
-					   &update->new_oid, &update->old_oid,
-					   NULL, NULL, NULL);
+		if (update->new_target || is_root_ref(update->refname)) {
+			if (!loose_transaction) {
+				loose_transaction = ref_store_transaction_begin(&refs->base, 0, err);
+				if (!loose_transaction) {
+					ret = TRANSACTION_GENERIC_ERROR;
+					goto cleanup;
+				}
+			}
+
+			ref_transaction_add_update(loose_transaction, update->refname,
+						   update->flags & ~REF_HAVE_OLD,
+						   update->new_target ? NULL : &update->new_oid, NULL,
+						   update->new_target, NULL, NULL);
+		} else {
+			ref_transaction_add_update(packed_transaction, update->refname,
+						   update->flags & ~REF_HAVE_OLD,
+						   &update->new_oid, &update->old_oid,
+						   NULL, NULL, NULL);
+		}
 	}
 
-	if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
+	if (packed_refs_lock(refs->packed_ref_store, 0, err) ||
+	    ref_transaction_commit(packed_transaction, err)) {
 		ret = TRANSACTION_GENERIC_ERROR;
 		goto cleanup;
 	}
+	packed_refs_unlock(refs->packed_ref_store);
 
-	if (ref_transaction_commit(packed_transaction, err)) {
-		ret = TRANSACTION_GENERIC_ERROR;
+	if (loose_transaction) {
+		if (ref_transaction_prepare(loose_transaction, err) ||
+		    ref_transaction_commit(loose_transaction, err)) {
+			ret = TRANSACTION_GENERIC_ERROR;
+			goto cleanup;
+		}
 	}
 
-	packed_refs_unlock(refs->packed_ref_store);
 cleanup:
+	if (loose_transaction)
+		ref_transaction_free(loose_transaction);
 	if (packed_transaction)
 		ref_transaction_free(packed_transaction);
 	transaction->state = REF_TRANSACTION_CLOSED;
-- 
2.47.0.274.g962d0b743d.dirty
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 5 of 12 in “refs: optimize ref format migrations”
  1. 00/10 refs: optimize ref format migrationsPatrick Steinhardt, Nov 25, 2024
  2. 01/10 refs: allow passing flags when setting up a transactionPatrick Steinhardt, Nov 25, 2024
  3. 02/10 refs/files: move logic to commit initial transactionPatrick Steinhardt, Nov 25, 2024
  4. 03/10 refs: introduce "initial" transaction flagPatrick Steinhardt, Nov 25, 2024
  5. 04/10 refs/files: support symbolic and root refs in initial transactionPatrick Steinhardt, Nov 25, 2024
  6. 05/10 refs: use "initial" transaction semantics to migrate refsPatrick Steinhardt, Nov 25, 2024
  7. 06/10 refs: skip collision checks in initial transactionsPatrick Steinhardt, Nov 25, 2024
  8. 07/10 refs: don't normalize log messages with `REF_SKIP_CREATE_REFLOG`Patrick Steinhardt, Nov 25, 2024
  9. 08/10 reftable/writer: optimize allocations by using a scratch bufferPatrick Steinhardt, Nov 25, 2024
  10. 10/10 reftable/block: optimize allocations by using scratch bufferPatrick Steinhardt, Nov 25, 2024
  11. 09/10 reftable/block: rename `block_writer::buf` variablePatrick Steinhardt, Nov 25, 2024
  12. Patrick SteinhardtNov 25, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.