git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 2/4] p5413: add perf tests for connectivity check

From
Kristofer Karlsson via GitGitGadget <gitgitgadget@gmail.com>
Date
Oct 12, 2026, 12:07 UTC
Message-ID
<211794c9252181fa0c31fcd968379c189d001a8c.1791806877.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.2211.v3.git.1791806877.gitgitgadget@gmail.com>
From: Kristofer Karlsson <krka@spotify.com>

Add benchmarks for the rev-list connectivity check. Each fixture varies one dimension: total tree size, incoming commit count, or files changed per commit.

Signed-off-by: Kristofer Karlsson <krka@spotify.com>
---
 ...enerate-repo-p5413-connectivity-check.perl | 44 ++++++++++
 t/perf/p5413-connectivity-check.sh            | 85 +++++++++++++++++++
 2 files changed, 129 insertions(+)
 create mode 100644 t/perf/generate-repo-p5413-connectivity-check.perl
 create mode 100755 t/perf/p5413-connectivity-check.sh
diff --git a/t/perf/generate-repo-p5413-connectivity-check.perl b/t/perf/generate-repo-p5413-connectivity-check.perl
new file mode 100644
index 0000000000..9546996280
--- /dev/null
+++ b/t/perf/generate-repo-p5413-connectivity-check.perl
@@ -0,0 +1,44 @@
+#!/usr/bin/perl
+#
+# Generate a fast-import stream for p5412 connectivity check benchmarks.
+#
+# Usage: generate-repo-p5412-connectivity-check.perl
+#            <dirs> <files_per_dir> <commits> [<hot_dirs>] [<files_per_commit>]
+#
+# Creates one initial commit with dirs*files_per_dir files, then
+# <commits> additional commits each modifying <files_per_commit>
+# files in directories chosen round-robin from 1..<hot_dirs>.
+
+use strict;
+use warnings;
+
+my ($nd, $nf, $nc, $hot, $fpc) = @ARGV;
+$hot = $nd if !$hot || $hot > $nd;
+$fpc = 1   if !$fpc;
+
+sub data {
+	printf "data %d\n%s\n", length($_[0]), $_[0];
+}
+
+# Initial tree: one commit with nd*nf files.
+printf "commit refs/heads/main\n";
+printf "committer perf <perf\@test.com> now\n";
+data("initial");
+for my $d (1..$nd) {
+	for my $f (1..$nf) {
+		printf "M 100644 inline d-%04d/f-%03d\n", $d, $f;
+		data(sprintf "%03d%03d", $d, $f);
+	}
+}
+
+# Subsequent commits (auto-chained by fast-import).
+for my $i (1..$nc) {
+	printf "commit refs/heads/main\n";
+	printf "committer perf <perf\@test.com> now\n";
+	data(sprintf "change-%03d", $i);
+	for my $j (0..$fpc-1) {
+		my $d = (($i + $j) % $hot) + 1;
+		printf "M 100644 inline d-%04d/f-001\n", $d;
+		data(sprintf "c%d-%d", $i, $j);
+	}
+}
diff --git a/t/perf/p5413-connectivity-check.sh b/t/perf/p5413-connectivity-check.sh
new file mode 100755
index 0000000000..3fabc82a87
--- /dev/null
+++ b/t/perf/p5413-connectivity-check.sh
@@ -0,0 +1,85 @@
+#!/bin/sh
+
+test_description='performance of connectivity check
+
+Benchmark the rev-list connectivity check, avoiding pack transfer
+noise.
+
+Each repository has a flat tree of many directories with 100 files
+in each.  Three axes are scaled independently: tree size, commit
+count, and files changed per commit.'
+
+. ./perf-lib.sh
+
+test_perf_fresh_repo
+
+generate="$TEST_DIRECTORY/perf/generate-repo-p5413-connectivity-check.perl"
+
+# $1=dirs  $2=files_per_dir  $3=commits  $4=hot_dirs (optional, default=all)
+# $5=files_per_commit (optional, default=1)
+test_perf_conn () {
+	local nd="$1" nf="$2" nc="$3" hot="${4:-$1}" fpc="${5:-1}"
+	local total=$(($nd * $nf))
+	local name="repo-${nd}d-${nf}f-${nc}c-${hot}h-${fpc}fpc"
+	local label="${total} files, ${nc} commits"
+	if test "$hot" -lt "$nd"
+	then
+		label="$label (${hot} hot dirs)"
+	fi
+	if test "$fpc" -gt 1
+	then
+		label="$label (${fpc} files/commit)"
+	fi
+
+	test_expect_success "setup $label" '
+		if test -d '"$name"'
+		then
+			true
+		else
+			git init '"$name"' &&
+			"$PERL_PATH" '"$generate"' '"$nd"' '"$nf"' '"$nc"' '"$hot"' '"$fpc"' |
+			git -C '"$name"' fast-import --date-format=now --quiet &&
+			(
+				cd '"$name"' &&
+				git rev-parse main~'"$nc"' >../'"${name}"'_old &&
+				git rev-parse main >../'"${name}"'_new &&
+				git update-ref refs/heads/main \
+					$(cat ../'"${name}"'_old) &&
+				git repack -ad &&
+				git config gc.auto 0
+			)
+		fi
+	'
+
+	test_perf "$label" '
+		cat '"${name}"'_new |
+		git -C '"$name"' rev-list \
+			--objects --stdin --not --all --quiet \
+			--exclude-promisor-objects
+	'
+}
+
+# Scaling tree size (10 commits, 10 files/commit).
+test_perf_conn   50 100 10   50 10
+test_perf_conn  500 100 10  500 10
+test_perf_conn 2000 100 10 2000 10
+test_perf_conn 8000 100 10 8000 10
+
+# Scaling commit count (200K files, 10 files/commit).
+test_perf_conn 2000 100    1 2000 10
+test_perf_conn 2000 100   10 2000 10
+test_perf_conn 2000 100  100 2000 10
+test_perf_conn 2000 100   500 2000 10
+test_perf_conn 2000 100  3000 2000 10
+test_perf_conn 2000 100  5000 2000 10
+test_perf_conn 2000 100 10000 2000 10
+
+# Scaling files per commit (200K files, 10 commits).
+test_perf_conn 2000 100 10 2000   1
+test_perf_conn 2000 100 10 2000  10
+test_perf_conn 2000 100 10 2000  100
+test_perf_conn 2000 100 10 2000  500
+test_perf_conn 2000 100 10 2000 1000
+test_perf_conn 2000 100 10 2000 2000
+
+test_done
-- 
gitgitgadget
Previous: Kristofer KarlssonNext: Patrick Steinhardt
Message 23 of 60 in “connected: add incremental connectivity check”
  1. 0/2 connected: add incremental connectivity checkKristofer Karlsson via GitGitGadget, Sep 14, 2026
  2. 1/2 Documentation: describe connectivity checkingKristofer Karlsson via GitGitGadget, Sep 14, 2026
  3. 2/2 connected: add incremental connectivity check via rev-listKristofer Karlsson via GitGitGadget, Sep 14, 2026
  4. Junio C HamanoSep 14, 2026
  5. Kristofer KarlssonSep 14, 2026
  6. Junio C HamanoSep 14, 2026
  7. 0/2 connected: add incremental connectivity checkKristofer Karlsson via GitGitGadget, Sep 28, 2026
  8. 1/2 Documentation: describe connectivity checkingKristofer Karlsson via GitGitGadget, Sep 28, 2026
  9. Patrick SteinhardtOct 5, 2026
  10. Junio C HamanoOct 5, 2026
  11. Patrick SteinhardtOct 6, 2026
  12. Kristofer KarlssonOct 6, 2026
  13. Kristofer KarlssonOct 6, 2026
  14. 2/2 connected: add incremental connectivity check via rev-listKristofer Karlsson via GitGitGadget, Sep 28, 2026
  15. Patrick SteinhardtOct 5, 2026
  16. Kristofer KarlssonOct 6, 2026
  17. Patrick SteinhardtOct 6, 2026
  18. Kristofer KarlssonOct 6, 2026
  19. 0/4 connected: add incremental connectivity checkKristofer Karlsson via GitGitGadget, Oct 12, 2026
  20. 1/4 Documentation: describe connectivity checkingKristofer Karlsson via GitGitGadget, Oct 12, 2026
  21. Patrick SteinhardtOct 15, 2026
  22. Kristofer KarlssonOct 15, 2026
  23. 2/4 p5413: add perf tests for connectivity checkKristofer Karlsson via GitGitGadget, Oct 12, 2026
  24. Patrick SteinhardtOct 15, 2026
  25. Kristofer KarlssonOct 15, 2026
  26. 3/4 connected: add transfer.connectivityCheck config and rev-list flagKristofer Karlsson via GitGitGadget, Oct 12, 2026
  27. Junio C HamanoOct 12, 2026
  28. Kristofer KarlssonOct 13, 2026
  29. Junio C HamanoOct 13, 2026
  30. Patrick SteinhardtOct 15, 2026
  31. Kristofer KarlssonOct 15, 2026
  32. 4/4 connected: add incremental connectivity check via rev-listKristofer Karlsson via GitGitGadget, Oct 12, 2026
  33. Patrick SteinhardtOct 15, 2026
  34. Kristofer KarlssonOct 15, 2026
  35. Patrick SteinhardtOct 15, 2026
  36. Kristofer KarlssonOct 16, 2026
  37. Patrick SteinhardtOct 19, 2026
  38. Junio C HamanoOct 12, 2026
  39. Kristofer KarlssonOct 13, 2026
  40. 0/6 connected: add incremental connectivity checkKristofer Karlsson via GitGitGadget, Oct 21, 2026
  41. 1/6 Documentation: describe connectivity checkingKristofer Karlsson via GitGitGadget, Oct 21, 2026
  42. 2/6 p5413: add perf tests for connectivity checkKristofer Karlsson via GitGitGadget, Oct 21, 2026
  43. 3/6 connected: drop dependency on `the_repository`Kristofer Karlsson via GitGitGadget, Oct 21, 2026
  44. 4/6 connected: add transfer.connectivityCheck configKristofer Karlsson via GitGitGadget, Oct 21, 2026
  45. 5/6 connected: add incremental connectivity check via rev-listKristofer Karlsson via GitGitGadget, Oct 21, 2026
  46. Junio C HamanoOct 21, 2026
  47. Kristofer KarlssonOct 22, 2026
  48. Kristofer KarlssonOct 22, 2026
  49. Junio C HamanoOct 22, 2026
  50. Kristofer KarlssonOct 22, 2026
  51. Junio C HamanoOct 23, 2026
  52. 6/6 connected: add "auto" connectivity check modeKristofer Karlsson via GitGitGadget, Oct 21, 2026
  53. 0/6 connected: add incremental connectivity checkKristofer Karlsson via GitGitGadget, Oct 23, 2026
  54. 1/6 Documentation: describe connectivity checkingKristofer Karlsson via GitGitGadget, Oct 23, 2026
  55. 2/6 p5413: add perf tests for connectivity checkKristofer Karlsson via GitGitGadget, Oct 23, 2026
  56. 3/6 connected: drop dependency on `the_repository`Kristofer Karlsson via GitGitGadget, Oct 23, 2026
  57. 4/6 connected: add transfer.connectivityCheck configKristofer Karlsson via GitGitGadget, Oct 23, 2026
  58. 5/6 connected: add incremental connectivity check via rev-listKristofer Karlsson via GitGitGadget, Oct 23, 2026
  59. 6/6 connected: add "auto" connectivity check modeKristofer Karlsson via GitGitGadget, Oct 23, 2026
  60. Junio C HamanoOct 23, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.