[PATCH v2 10/17] p5313: add performance tests for --path-walk

From: Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date: Oct 20, 2024, 13:43 UTC
Message-ID: <21dc3723c365f28d7e31db15044744060a977a3d.1729431810.git.gitgitgadget@gmail.com>
In-Reply-To: <pull.1813.v2.git.1729431810.gitgitgadget@gmail.com>

From: Derrick Stolee <stolee@gmail.com>

The previous change added a --path-walk option to 'git pack-objects'. Create a performance test that demonstrates the time and space benefits of the feature.

In order to get an appropriate comparison, we need to avoid reusing deltas and recompute them from scratch.

Compare the creation of a thin pack representing a small push and the creation of a relatively large non-thin pack.

Running on my copy of the Git repository results in this data:

Test this tree --------------------------------------------------------- 5313.2: thin pack 0.01(0.00+0.00) 5313.3: thin pack size 1.1K 5313.4: thin pack with --path-walk 0.01(0.01+0.00) 5313.5: thin pack size with --path-walk 1.1K 5313.6: big pack 2.52(6.59+0.38) 5313.7: big pack size 14.1M 5313.8: big pack with --path-walk 4.90(5.76+0.26) 5313.9: big pack size with --path-walk 13.2M

Note that the timing is slower because there is no threading in the --path-walk case (yet).

The cases where the --path-walk option really shines is when the default name-hash is overwhelmed with collisions. An open source example can be found in the microsoft/fluentui repo [1] at a certain commit [2].

[1] https://github.com/microsoft/fluentui [2] e70848ebac1cd720875bccaa3026f4a9ed700e08

Running the tests on this repo results in the following output:

Test this tree ---------------------------------------------------------- 5313.2: thin pack 0.28(0.38+0.02) 5313.3: thin pack size 1.2M 5313.4: thin pack with --path-walk 0.08(0.06+0.01) 5313.5: thin pack size with --path-walk 18.4K 5313.6: big pack 4.05(29.62+0.43) 5313.7: big pack size 20.0M 5313.8: big pack with --path-walk 5.99(9.06+0.24) 5313.9: big pack size with --path-walk 16.4M

Notice in particular that in the small thin pack, the time performance has improved from 0.28s to 0.08s and this is likely due to the improved size of the resulting pack: 18.4K instead of 1.2M.

Finally, running this on a copy of the Linux kernel repository results in these data points:

Test this tree ----------------------------------------------------------- 5313.2: thin pack 0.00(0.00+0.00) 5313.3: thin pack size 5.8K 5313.4: thin pack with --path-walk 0.00(0.01+0.00) 5313.5: thin pack size with --path-walk 5.8K 5313.6: big pack 24.39(65.81+1.31) 5313.7: big pack size 155.7M 5313.8: big pack with --path-walk 41.07(60.69+0.68) 5313.9: big pack size with --path-walk 150.8M

Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
 t/perf/p5313-pack-objects.sh | 59 ++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)
 create mode 100755 t/perf/p5313-pack-objects.sh

diff --git a/t/perf/p5313-pack-objects.sh b/t/perf/p5313-pack-objects.sh
new file mode 100755
index 00000000000..840075f5691
--- /dev/null
+++ b/t/perf/p5313-pack-objects.sh
@@ -0,0 +1,59 @@
+#!/bin/sh
+
+test_description='Tests pack performance using bitmaps'
+. ./perf-lib.sh
+
+GIT_TEST_PASSING_SANITIZE_LEAK=0
+export GIT_TEST_PASSING_SANITIZE_LEAK
+
+test_perf_large_repo
+
+test_expect_success 'create rev input' '
+	cat >in-thin <<-EOF &&
+	$(git rev-parse HEAD)
+	^$(git rev-parse HEAD~1)
+	EOF
+
+	cat >in-big <<-EOF
+	$(git rev-parse HEAD)
+	^$(git rev-parse HEAD~1000)
+	EOF
+'
+
+test_perf 'thin pack' '
+	git pack-objects --thin --stdout --no-reuse-delta \
+		--revs --sparse <in-thin >out
+'
+
+test_size 'thin pack size' '
+	test_file_size out
+'
+
+test_perf 'thin pack with --path-walk' '
+	git pack-objects --thin --stdout --no-reuse-delta \
+		--revs --sparse --path-walk <in-thin >out
+'
+
+test_size 'thin pack size with --path-walk' '
+	test_file_size out
+'
+
+test_perf 'big pack' '
+	git pack-objects --stdout --no-reuse-delta --revs \
+		--sparse <in-big >out
+'
+
+test_size 'big pack size' '
+	test_file_size out
+'
+
+test_perf 'big pack with --path-walk' '
+	git pack-objects --stdout --no-reuse-delta --revs \
+		--sparse --path-walk <in-big >out
+'
+
+test_size 'big pack size with --path-walk' '
+	test_file_size out
+'
+
+test_done

-- 
gitgitgadget