git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 07/10] refs: don't normalize log messages with `REF_SKIP_CREATE_REFLOG`

From
Patrick Steinhardt <ps@pks.im>
Date
Nov 8, 2024, 09:34 UTC
Message-ID
<20241108-pks-refs-optimize-migrations-v1-7-7fd37fa80e35@pks.im>
In-Reply-To
<20241108-pks-refs-optimize-migrations-v1-0-7fd37fa80e35@pks.im>

When the `REF_SKIP_CREATE_REFLOG` flag is set we skip the creation of the reflog entry, but we still normalize the reflog message when we queue the update. This is a waste of resources as the normalized message will never get used in the first place.

Fix this issue by skipping the normalization in case the flag is set. This leads to a surprisingly large speedup when migrating from the "files" to the "reftable" backend:

    Benchmark 1: migrate files:reftable (refcount = 1000000, revision = HEAD~)
      Time (mean ± σ):     878.5 ms ±  14.9 ms    [User: 726.5 ms, System: 139.2 ms]
      Range (min … max):   858.4 ms … 941.3 ms    50 runs
    Benchmark 2: migrate files:reftable (refcount = 1000000, revision = HEAD)
      Time (mean ± σ):     831.1 ms ±  10.5 ms    [User: 694.1 ms, System: 126.3 ms]
      Range (min … max):   812.4 ms … 851.4 ms    50 runs
    Summary
      migrate files:reftable (refcount = 1000000, revision = HEAD) ran
        1.06 ± 0.02 times faster than migrate files:reftable (refcount = 1000000, revision = HEAD~)
And an ever larger speedup when migrating the other way round:
    Benchmark 1: migrate reftable:files (refcount = 1000000, revision = HEAD~)
      Time (mean ± σ):     923.6 ms ±  11.6 ms    [User: 705.5 ms, System: 208.1 ms]
      Range (min … max):   905.3 ms … 946.5 ms    50 runs
    Benchmark 2: migrate reftable:files (refcount = 1000000, revision = HEAD)
      Time (mean ± σ):     818.5 ms ±   9.0 ms    [User: 627.6 ms, System: 180.6 ms]
      Range (min … max):   802.2 ms … 842.9 ms    50 runs
    Summary
      migrate reftable:files (refcount = 1000000, revision = HEAD) ran
        1.13 ± 0.02 times faster than migrate reftable:files (refcount = 1000000, revision = HEAD~)
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 refs.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/refs.c b/refs.c
index d690eb19b3fd7083a5309deb98738547e4f48040..65eea3eb7734d03f09a22e8edfe5074d532398ff 100644
--- a/refs.c
+++ b/refs.c
@@ -1188,8 +1188,9 @@ struct ref_update *ref_transaction_add_update(
 		oidcpy(&update->new_oid, new_oid);
 	if ((flags & REF_HAVE_OLD) && old_oid)
 		oidcpy(&update->old_oid, old_oid);
+	if (!(flags & REF_SKIP_CREATE_REFLOG))
+		update->msg = normalize_reflog_message(msg);
 
-	update->msg = normalize_reflog_message(msg);
 	return update;
 }
 
-- 
2.47.0.229.g8f8d6eee53.dirty
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 16 of 42 in “refs: optimize ref format migrations”
  1. 00/10 refs: optimize ref format migrationsPatrick Steinhardt, Nov 8, 2024
  2. 01/10 refs: allow passing flags when setting up a transactionPatrick Steinhardt, Nov 8, 2024
  3. karthik nayakNov 11, 2024
  4. Patrick SteinhardtNov 11, 2024
  5. 02/10 refs/files: move logic to commit initial transactionPatrick Steinhardt, Nov 8, 2024
  6. 03/10 refs: introduce "initial" transaction flagPatrick Steinhardt, Nov 8, 2024
  7. 05/10 refs: use "initial" transaction semantics to migrate refsPatrick Steinhardt, Nov 8, 2024
  8. karthik nayakNov 11, 2024
  9. 06/10 refs: skip collision checks in initial transactionsPatrick Steinhardt, Nov 8, 2024
  10. karthik nayakNov 11, 2024
  11. 04/10 refs/files: support symbolic and root refs in initial transactionPatrick Steinhardt, Nov 8, 2024
  12. karthik nayakNov 11, 2024
  13. Patrick SteinhardtNov 11, 2024
  14. 09/10 reftable/block: rename `block_writer::buf` variablePatrick Steinhardt, Nov 8, 2024
  15. 08/10 reftable/writer: optimize allocations by using a scratch bufferPatrick Steinhardt, Nov 8, 2024
  16. 07/10 refs: don't normalize log messages with `REF_SKIP_CREATE_REFLOG`Patrick Steinhardt, Nov 8, 2024
  17. 10/10 reftable/block: optimize allocations by using scratch bufferPatrick Steinhardt, Nov 8, 2024
  18. karthik nayakNov 11, 2024
  19. Patrick SteinhardtNov 11, 2024
  20. Junio C HamanoNov 20, 2024
  21. Patrick SteinhardtNov 20, 2024
  22. Christian CouderNov 20, 2024
  23. Patrick SteinhardtNov 25, 2024
  24. 00/10 refs: optimize ref format migrationsPatrick Steinhardt, Nov 20, 2024
  25. 01/10 refs: allow passing flags when setting up a transactionPatrick Steinhardt, Nov 20, 2024
  26. Christian CouderNov 20, 2024
  27. 02/10 refs/files: move logic to commit initial transactionPatrick Steinhardt, Nov 20, 2024
  28. 03/10 refs: introduce "initial" transaction flagPatrick Steinhardt, Nov 20, 2024
  29. 04/10 refs/files: support symbolic and root refs in initial transactionPatrick Steinhardt, Nov 20, 2024
  30. 05/10 refs: use "initial" transaction semantics to migrate refsPatrick Steinhardt, Nov 20, 2024
  31. 06/10 refs: skip collision checks in initial transactionsPatrick Steinhardt, Nov 20, 2024
  32. Christian CouderNov 20, 2024
  33. Patrick SteinhardtNov 25, 2024
  34. Kristoffer HaugsbakkNov 20, 2024
  35. Patrick SteinhardtNov 25, 2024
  36. 07/10 refs: don't normalize log messages with `REF_SKIP_CREATE_REFLOG`Patrick Steinhardt, Nov 20, 2024
  37. 08/10 reftable/writer: optimize allocations by using a scratch bufferPatrick Steinhardt, Nov 20, 2024
  38. Christian CouderNov 20, 2024
  39. Patrick SteinhardtNov 25, 2024
  40. 09/10 reftable/block: rename `block_writer::buf` variablePatrick Steinhardt, Nov 20, 2024
  41. 10/10 reftable/block: optimize allocations by using scratch bufferPatrick Steinhardt, Nov 20, 2024
  42. Christian CouderNov 20, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.