git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 7/7] t5551: use reftables for repositories with many refs

From
Patrick Steinhardt <ps@pks.im>
Date
Nov 11, 2026, 10:47 UTC
Message-ID
<20261111-pks-gitlab-ci-speedups-v3-7-d1e34117ee2e@pks.im>
In-Reply-To
<20261111-pks-gitlab-ci-speedups-v3-0-d1e34117ee2e@pks.im>

Some of the tests in t5551 creates almost a 100,000 references and then verify that those can be fetched. Naturally, creating this many refs is quite involved with the "files" backend, as it means that we have to create this many individual loose files. This is tolerable on Linux, where the test takes around 2 minutes in our CI. But on macOS, the same test takes almost 20 minutes to complete. And to add insult to injury, we execute these tests a second time via t5559.

The system under test here is the HTTP negotiation, so we don't actually care about whether or not the ref backend handles this alright. So adapt the test to instead create repositories with the reftable backend, which is significantly faster in this case as we only need to write a single table.

And sure enough, this reduces the runtime of this test on macOS by a factor of 10, going from 20 minutes to 2 minutes. On Linux, the tests at least speed up by a factor of two.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t5551-http-fetch-smart.sh | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh
index 130053c57aa..34945059531 100755
--- a/t/t5551-http-fetch-smart.sh
+++ b/t/t5551-http-fetch-smart.sh
@@ -412,7 +412,8 @@ create_tags () {
 }
 
 test_expect_success 'create 2,000 tags in the repo' '
-	git init --bare "$HTTPD_DOCUMENT_ROOT_PATH/many-tags.git" &&
+	git init --bare --ref-format=reftable \
+		"$HTTPD_DOCUMENT_ROOT_PATH/many-tags.git" &&
 	(
 		cd "$HTTPD_DOCUMENT_ROOT_PATH/many-tags.git" &&
 		create_tags 1 2000
@@ -421,7 +422,8 @@ test_expect_success 'create 2,000 tags in the repo' '
 
 test_expect_success CMDLINE_LIMIT \
 	'clone the 2,000 tag repo to check OS command line overflow' '
-	run_with_limited_cmdline git clone $HTTPD_URL/smart/many-tags.git too-many-refs &&
+	run_with_limited_cmdline git clone --ref-format=reftable \
+		$HTTPD_URL/smart/many-tags.git too-many-refs &&
 	(
 		cd too-many-refs &&
 		git for-each-ref refs/tags >actual &&
-- 
2.56.0.611.g1030f0b940.dirty
Previous: Patrick Steinhardt
Message 40 of 40 in “ci: some speedups for slow tests”
  1. 0/7 ci: some speedups for slow testsPatrick Steinhardt, Nov 6, 2026
  2. 1/7 gitlab-ci: speed up installation of Git for Windows SDKPatrick Steinhardt, Nov 6, 2026
  3. Junio C HamanoNov 6, 2026
  4. Patrick SteinhardtNov 9, 2026
  5. 2/7 gitlab-ci: exclude intermediate outputs from Windows artifactsPatrick Steinhardt, Nov 6, 2026
  6. 3/7 ci: build optimized binaries with MesonPatrick Steinhardt, Nov 6, 2026
  7. Junio C HamanoNov 6, 2026
  8. 4/7 t0027: optimize how we reset repository statePatrick Steinhardt, Nov 6, 2026
  9. Junio C HamanoNov 6, 2026
  10. Torsten BögershausenNov 8, 2026
  11. Patrick SteinhardtNov 9, 2026
  12. 5/7 t0027: cache expected output filesPatrick Steinhardt, Nov 6, 2026
  13. 6/7 t1092: set up repositories from templatesPatrick Steinhardt, Nov 6, 2026
  14. 7/7 t5551: use reftables for repositories with many refsPatrick Steinhardt, Nov 6, 2026
  15. Junio C HamanoNov 6, 2026
  16. 0/7 ci: some speedups for slow testsPatrick Steinhardt, Nov 9, 2026
  17. 1/7 gitlab-ci: speed up installation of Git for Windows SDKPatrick Steinhardt, Nov 9, 2026
  18. Junio C HamanoNov 9, 2026
  19. Patrick SteinhardtNov 10, 2026
  20. Junio C HamanoNov 10, 2026
  21. 2/7 gitlab-ci: exclude intermediate outputs from Windows artifactsPatrick Steinhardt, Nov 9, 2026
  22. 3/7 ci: build optimized binaries with MesonPatrick Steinhardt, Nov 9, 2026
  23. 4/7 t0027: optimize how we reset repository statePatrick Steinhardt, Nov 9, 2026
  24. Junio C HamanoNov 9, 2026
  25. Patrick SteinhardtNov 10, 2026
  26. 5/7 t0027: cache expected output filesPatrick Steinhardt, Nov 9, 2026
  27. Torsten BögershausenNov 9, 2026
  28. Patrick SteinhardtNov 10, 2026
  29. Torsten BögershausenNov 10, 2026
  30. 6/7 t1092: set up repositories from templatesPatrick Steinhardt, Nov 9, 2026
  31. 7/7 t5551: use reftables for repositories with many refsPatrick Steinhardt, Nov 9, 2026
  32. Johannes SixtNov 12, 2026
  33. 0/7 ci: some speedups for slow testsPatrick Steinhardt, Nov 11, 2026
  34. 1/7 gitlab-ci: speed up installation of Git for Windows SDKPatrick Steinhardt, Nov 11, 2026
  35. 2/7 gitlab-ci: exclude intermediate outputs from Windows artifactsPatrick Steinhardt, Nov 11, 2026
  36. 3/7 ci: build optimized binaries with MesonPatrick Steinhardt, Nov 11, 2026
  37. 4/7 t0027: optimize how we reset repository statePatrick Steinhardt, Nov 11, 2026
  38. 5/7 t0027: cache expected output filesPatrick Steinhardt, Nov 11, 2026
  39. 6/7 t1092: set up repositories from templatesPatrick Steinhardt, Nov 11, 2026
  40. 7/7 t5551: use reftables for repositories with many refsPatrick Steinhardt, Nov 11, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.