git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 3/3] t5551: pack refs after creating many tags

From
Patrick Steinhardt <ps@pks.im>
Date
Jun 30, 2026, 09:05 UTC
Message-ID
<akOG0oMu2KTqqyW7@pks.im>
In-Reply-To
<20260629203527.GA1895313@coredump.intra.peff.net>
On Mon, Jun 29, 2026 at 04:35:27PM -0400, Jeff King wrote:
Show 12 quoted lines
> There's one other thing you might find interesting. While poking at the
> timings here the other day, I noticed that reftable is very eager to
> stat the tables.list file. Try this:
> 
>   git init --ref-format=reftable
>   blob=$(echo foo | git hash-object -w --stdin)
>   seq -f "create refs/tags/foo-%g $blob" 2000 |
>   strace -c git update-ref --stdin
> 
> We make 2000 fstat, which strace claims takes 85% of the time. I suspect
> this is over-emphasized because strace inherently makes syscalls slow,
> but running with perf also highlights it as a non-trivial cost.

Yeah, this rings a bell. If I remember correctly, this is because we call `refs_resolve_ref_unsafe()` to verify whether the target already exists. And as that function is generic, it wasn't easy to optimize it by reusing the already-loaded reftable stack.

> It has been a long time since I've thought about reftable internals, but
> it feels like we ought to be able to take the lock and then trust that
> the stack has not been manipulated.
Yeah, that would certainly be an option to explore.
> It may not be worth digging into too much, though. I can make 50,000
> refs in 150ms on my system, which is probably good enough (especially
> compared to the files backend).

True, it's going to be much better compared to the "files" backend. But that isn't enough reason to not optimize it even further -- doubly so if it actually takes 85% of the time. Sounds like a low-hanging fruit to me that can result in a significant speedup.

I'll probably not get to it anytime soon, but I'll create an issue to keep track of it.

Patrick
Previous: Jeff KingNext: Jeff King
Message 24 of 42 in “Re: [RFH] Why do osx CI jobs so unreliable?”
  1. Michael MontalboJun 20, 2026
  2. Jeff KingJun 21, 2026
  3. Patrick SteinhardtJun 22, 2026
  4. Patrick SteinhardtJun 22, 2026
  5. Patrick SteinhardtJun 22, 2026
  6. Patrick SteinhardtJun 22, 2026
  7. Michael MontalboJun 26, 2026
  8. Jeff KingJun 26, 2026
  9. Patrick SteinhardtJun 26, 2026
  10. Junio C HamanoJun 26, 2026
  11. Michael MontalboJun 26, 2026
  12. 0/3 fixing expensive http test timeoutsJeff King, Jun 28, 2026
  13. 1/3 t/lib-httpd: bump apache timeoutJeff King, Jun 28, 2026
  14. Michael MontalboJul 2, 2026
  15. 2/3 t5551: put many-tags case into its own repoJeff King, Jun 28, 2026
  16. Junio C HamanoJun 28, 2026
  17. Jeff KingJun 29, 2026
  18. Junio C HamanoJun 29, 2026
  19. Jeff KingJun 29, 2026
  20. 3/3 t5551: pack refs after creating many tagsJeff King, Jun 28, 2026
  21. Junio C HamanoJun 28, 2026
  22. Patrick SteinhardtJun 29, 2026
  23. Jeff KingJun 29, 2026
  24. Patrick SteinhardtJun 30, 2026
  25. Jeff KingJun 30, 2026
  26. weird quadratic reftable behavior, was: Re: [PATCH 3/3] t5551: pack refs after creating many tagsJeff King, Jun 30, 2026
  27. Patrick SteinhardtJul 1, 2026
  28. Jeff KingJul 1, 2026
  29. Kristofer KarlssonJul 1, 2026
  30. Patrick SteinhardtJul 1, 2026
  31. Kristofer KarlssonJul 3, 2026
  32. Patrick SteinhardtJul 6, 2026
  33. Kristofer KarlssonJul 6, 2026
  34. Patrick SteinhardtJun 29, 2026
  35. Junio C HamanoJun 29, 2026
  36. Patrick SteinhardtJun 29, 2026
  37. Junio C HamanoJun 29, 2026
  38. Patrick SteinhardtJun 30, 2026
  39. Junio C HamanoJun 30, 2026
  40. Patrick SteinhardtJul 1, 2026
  41. Jeff KingJun 26, 2026
  42. Junio C HamanoJun 22, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.