git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 3/3] t5551: pack refs after creating many tags

From
Jeff King <peff@peff.net>
Date
Jun 28, 2026, 08:07 UTC
Message-ID
<20260628080710.GC107826@coredump.intra.peff.net>
In-Reply-To
<20260628075716.GA3525066@coredump.intra.peff.net>

We have two tests that create 2,000 and 100,000 tags respectively. After doing so, the resulting state can be a bit slow to work with when using the "files" ref backend, as each of those refs is in its own file.

This isn't a very realistic scenario, as we'd expect most of those refs to be packed. If they accrue over time along with objects, they'd get packed by maintenance/gc runs. And if you have a process that creates a ton of refs at once (like a big fast-import), the usual recommendation is to run maintenance afterwards.

So let's follow that recommendation and pack the refs ourselves. Unfortunately, this does not seem to produce an improvement to the run-time of the test script! That's because after producing this state, we perform only a few fetches of it. And packing the refs costs at least as much as serving a ref advertisement (both have to iterate the refs, but packing additionally must write .lock files as we pack).

My wall-clock time was slightly improved (but within the noise) with this patch, but my user and system CPU time were slightly worse! However, on a loaded system with I/O bottlenecks, it may be a net win. That's somewhat of a guess, though.

It would be nice if we had a way to generate all of these refs without writing so many individual files. But even if we taught the ref code to write large cases directly to the packed-refs file, we'd still need to take individual locks. The real solution is a backend like reftable, which shaves ~30% off of the test runtime.

Signed-off-by: Jeff King <peff@peff.net>
---
I'm iffy on whether this one is worth it.

If you apply just this patch without patch 2, then the run-time does improve quite a bit. The cost of packing is amortized by the improved performance for all of those subsequent tests (but after patch 2, they never even see the unpacked state).

Likewise, I suspect this would make our timeout problems go away even without patch 1.

So the whole series _could_ be reduced to just this one patch. But hopefully the reasoning given in the earlier patches makes sense, at which point this one is kind of superfluous.

 t/t5551-http-fetch-smart.sh | 1 +
 1 file changed, 1 insertion(+)
diff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh
index cd851f24b8..e2e729216f 100755
--- a/t/t5551-http-fetch-smart.sh
+++ b/t/t5551-http-fetch-smart.sh
@@ -393,6 +393,7 @@ create_tags () {
 	tag=$(perl -e "print \"bla\" x 30") &&
 	sed -e "s|^:\([^ ]*\) \(.*\)$|create refs/tags/$tag-\1 \2|" <marks >input &&
 	git update-ref --stdin <input &&
+	git pack-refs --all &&
 	rm input
 }
 
-- 
2.55.0.rc2.353.gf769b6597e
Previous: Jeff KingNext: Junio C Hamano
Message 20 of 42 in “Re: [RFH] Why do osx CI jobs so unreliable?”
  1. Michael MontalboJun 20, 2026
  2. Jeff KingJun 21, 2026
  3. Patrick SteinhardtJun 22, 2026
  4. Patrick SteinhardtJun 22, 2026
  5. Patrick SteinhardtJun 22, 2026
  6. Patrick SteinhardtJun 22, 2026
  7. Michael MontalboJun 26, 2026
  8. Jeff KingJun 26, 2026
  9. Patrick SteinhardtJun 26, 2026
  10. Junio C HamanoJun 26, 2026
  11. Michael MontalboJun 26, 2026
  12. 0/3 fixing expensive http test timeoutsJeff King, Jun 28, 2026
  13. 1/3 t/lib-httpd: bump apache timeoutJeff King, Jun 28, 2026
  14. Michael MontalboJul 2, 2026
  15. 2/3 t5551: put many-tags case into its own repoJeff King, Jun 28, 2026
  16. Junio C HamanoJun 28, 2026
  17. Jeff KingJun 29, 2026
  18. Junio C HamanoJun 29, 2026
  19. Jeff KingJun 29, 2026
  20. 3/3 t5551: pack refs after creating many tagsJeff King, Jun 28, 2026
  21. Junio C HamanoJun 28, 2026
  22. Patrick SteinhardtJun 29, 2026
  23. Jeff KingJun 29, 2026
  24. Patrick SteinhardtJun 30, 2026
  25. Jeff KingJun 30, 2026
  26. weird quadratic reftable behavior, was: Re: [PATCH 3/3] t5551: pack refs after creating many tagsJeff King, Jun 30, 2026
  27. Patrick SteinhardtJul 1, 2026
  28. Jeff KingJul 1, 2026
  29. Kristofer KarlssonJul 1, 2026
  30. Patrick SteinhardtJul 1, 2026
  31. Kristofer KarlssonJul 3, 2026
  32. Patrick SteinhardtJul 6, 2026
  33. Kristofer KarlssonJul 6, 2026
  34. Patrick SteinhardtJun 29, 2026
  35. Junio C HamanoJun 29, 2026
  36. Patrick SteinhardtJun 29, 2026
  37. Junio C HamanoJun 29, 2026
  38. Patrick SteinhardtJun 30, 2026
  39. Junio C HamanoJun 30, 2026
  40. Patrick SteinhardtJul 1, 2026
  41. Jeff KingJun 26, 2026
  42. Junio C HamanoJun 22, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.