git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 05/10] refs: use "initial" transaction semantics to migrate refs

From
karthik nayak <karthik.188@gmail.com>
Date
Nov 11, 2024, 10:43 UTC
Message-ID
<CAOLa=ZSOVYKgwkm-4w6cr_SSpGTJ2QMC-5pa3xm3mR5sBWmYMQ@mail.gmail.com>
In-Reply-To
<20241108-pks-refs-optimize-migrations-v1-5-7fd37fa80e35@pks.im>
Patrick Steinhardt <ps@pks.im> writes:
Show 27 quoted lines
> Until now, we couldn't use "initial" transaction semantics to migrate
> refs because the "files" backend only supported writing regular refs via
> the initial transaction because it simply mapped the transaction to a
> "packed-refs" transaction. But with the preceding commit, the "files"
> backend has learned to also write symbolic and root refs in the initial
> transaction by creating a second transaction for all refs that need to
> be written as loose refs.
>
> Adapt the code to migrate refs to commit the transaction as an initial
> transaction. This results in a signiticant speedup when migrating many
> refs:
>
>     Benchmark 1: migrate reftable:files (refcount = 100000, revision = HEAD~)
>       Time (mean ± σ):      3.247 s ±  0.034 s    [User: 0.485 s, System: 2.722 s]
>       Range (min … max):    3.216 s …  3.309 s    10 runs
>
>     Benchmark 2: migrate reftable:files (refcount = 100000, revision = HEAD)
>       Time (mean ± σ):     453.6 ms ±   1.9 ms    [User: 214.6 ms, System: 230.5 ms]
>       Range (min … max):   451.5 ms … 456.4 ms    10 runs
>
>     Summary
>       migrate reftable:files (refcount = 100000, revision = HEAD) ran
>         7.16 ± 0.08 times faster than migrate reftable:files (refcount = 100000, revision = HEAD~)
>
> As the reftable backend doesn't (yet) special-case initial transactions
> there is no comparable speedup for that backend.
>

Nice. Really like how the actual perf changes were built up to a single line change.

[snip]
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 8 of 42 in “refs: optimize ref format migrations”
  1. 00/10 refs: optimize ref format migrationsPatrick Steinhardt, Nov 8, 2024
  2. 01/10 refs: allow passing flags when setting up a transactionPatrick Steinhardt, Nov 8, 2024
  3. karthik nayakNov 11, 2024
  4. Patrick SteinhardtNov 11, 2024
  5. 02/10 refs/files: move logic to commit initial transactionPatrick Steinhardt, Nov 8, 2024
  6. 03/10 refs: introduce "initial" transaction flagPatrick Steinhardt, Nov 8, 2024
  7. 05/10 refs: use "initial" transaction semantics to migrate refsPatrick Steinhardt, Nov 8, 2024
  8. karthik nayakNov 11, 2024
  9. 06/10 refs: skip collision checks in initial transactionsPatrick Steinhardt, Nov 8, 2024
  10. karthik nayakNov 11, 2024
  11. 04/10 refs/files: support symbolic and root refs in initial transactionPatrick Steinhardt, Nov 8, 2024
  12. karthik nayakNov 11, 2024
  13. Patrick SteinhardtNov 11, 2024
  14. 09/10 reftable/block: rename `block_writer::buf` variablePatrick Steinhardt, Nov 8, 2024
  15. 08/10 reftable/writer: optimize allocations by using a scratch bufferPatrick Steinhardt, Nov 8, 2024
  16. 07/10 refs: don't normalize log messages with `REF_SKIP_CREATE_REFLOG`Patrick Steinhardt, Nov 8, 2024
  17. 10/10 reftable/block: optimize allocations by using scratch bufferPatrick Steinhardt, Nov 8, 2024
  18. karthik nayakNov 11, 2024
  19. Patrick SteinhardtNov 11, 2024
  20. Junio C HamanoNov 20, 2024
  21. Patrick SteinhardtNov 20, 2024
  22. Christian CouderNov 20, 2024
  23. Patrick SteinhardtNov 25, 2024
  24. 00/10 refs: optimize ref format migrationsPatrick Steinhardt, Nov 20, 2024
  25. 01/10 refs: allow passing flags when setting up a transactionPatrick Steinhardt, Nov 20, 2024
  26. Christian CouderNov 20, 2024
  27. 02/10 refs/files: move logic to commit initial transactionPatrick Steinhardt, Nov 20, 2024
  28. 03/10 refs: introduce "initial" transaction flagPatrick Steinhardt, Nov 20, 2024
  29. 04/10 refs/files: support symbolic and root refs in initial transactionPatrick Steinhardt, Nov 20, 2024
  30. 05/10 refs: use "initial" transaction semantics to migrate refsPatrick Steinhardt, Nov 20, 2024
  31. 06/10 refs: skip collision checks in initial transactionsPatrick Steinhardt, Nov 20, 2024
  32. Christian CouderNov 20, 2024
  33. Patrick SteinhardtNov 25, 2024
  34. Kristoffer HaugsbakkNov 20, 2024
  35. Patrick SteinhardtNov 25, 2024
  36. 07/10 refs: don't normalize log messages with `REF_SKIP_CREATE_REFLOG`Patrick Steinhardt, Nov 20, 2024
  37. 08/10 reftable/writer: optimize allocations by using a scratch bufferPatrick Steinhardt, Nov 20, 2024
  38. Christian CouderNov 20, 2024
  39. Patrick SteinhardtNov 25, 2024
  40. 09/10 reftable/block: rename `block_writer::buf` variablePatrick Steinhardt, Nov 20, 2024
  41. 10/10 reftable/block: optimize allocations by using scratch bufferPatrick Steinhardt, Nov 20, 2024
  42. Christian CouderNov 20, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.