Re: [PATCH 0/3] pack-objects: support bitmaps and delta-islands with `--path-walk`
- From
Derrick Stolee <stolee@gmail.com>
- Date
- May 29, 2026, 17:26 UTC
- Message-ID
- <22a7e32f-f645-4f00-bc5b-6b4309e483c2@gmail.com>
- In-Reply-To
- <a708e23d-e0c2-48c9-86e9-1227f12edd53@gmail.com>
On 5/28/26 11:28 AM, Derrick Stolee wrote:
> On 5/27/26 7:18 PM, Taylor Blau wrote:
Show 5 quoted lines
> Do you have any end-to-end performance data to demonstrate that these > changes are effective at scale? Are we still producing packfiles with the > pack-file compression and now with .bitmap files? How does this impact > the performance of a clone or fetch when using a bitmap index at read > time?
Here's my attempt to use our existing performance tests to analyze the impact of this series.
Running p5311 against the base of this topic and this topic with GIT_TEST_PACK_PATH_WALK=1, I get this output:
Test HEAD~3 HEAD ----------------------------------------------------------------- 5311.4: server (1 days) (lookup=true) 0.02 0.03 +50.0% 5311.5: size (1 days) 6.8K 124.9K +1730.9% 5311.6: client (1 days) (lookup=true) 0.02 0.01 -50.0% 5311.8: server (2 days) (lookup=true) 0.02 0.03 +50.0% 5311.9: size (2 days) 6.8K 124.9K +1730.9% 5311.10: client (2 days) (lookup=true) 0.02 0.01 -50.0% 5311.12: server (4 days) (lookup=true) 0.02 0.03 +50.0% 5311.13: size (4 days) 6.8K 124.9K +1730.9% 5311.14: client (4 days) (lookup=true) 0.02 0.01 -50.0% 5311.16: server (8 days) (lookup=true) 0.03 0.03 +0.0% 5311.17: size (8 days) 37.3K 186.0K +398.2% 5311.18: client (8 days) (lookup=true) 0.03 0.02 -33.3% 5311.20: server (16 days) (lookup=true) 0.02 0.03 +50.0% 5311.21: size (16 days) 37.3K 186.0K +398.2% 5311.22: client (16 days) (lookup=true) 0.03 0.02 -33.3% 5311.24: server (32 days) (lookup=true) 0.03 0.03 +0.0% 5311.25: size (32 days) 46.5K 197.2K +324.3% 5311.26: client (32 days) (lookup=true) 0.03 0.02 -33.3% 5311.28: server (64 days) (lookup=true) 0.24 0.16 -33.3% 5311.29: size (64 days) 1.5M 5.1M +239.8% 5311.30: client (64 days) (lookup=true) 0.42 0.35 -16.7% 5311.32: server (128 days) (lookup=true) 0.49 0.29 -40.8% 5311.33: size (128 days) 4.1M 9.8M +139.5% 5311.34: client (128 days) (lookup=true) 0.86 0.65 -24.4% 5311.38: server (1 days) (lookup=false) 0.02 0.03 +50.0% 5311.39: size (1 days) 6.8K 124.9K +1730.9% 5311.40: client (1 days) (lookup=false) 0.02 0.02 +0.0% 5311.42: server (2 days) (lookup=false) 0.02 0.03 +50.0% 5311.43: size (2 days) 6.8K 124.9K +1730.9% 5311.44: client (2 days) (lookup=false) 0.02 0.02 +0.0% 5311.46: server (4 days) (lookup=false) 0.02 0.03 +50.0% 5311.47: size (4 days) 6.8K 124.9K +1730.9% 5311.48: client (4 days) (lookup=false) 0.02 0.02 +0.0% 5311.50: server (8 days) (lookup=false) 0.02 0.03 +50.0% 5311.51: size (8 days) 37.3K 186.0K +398.2% 5311.52: client (8 days) (lookup=false) 0.03 0.02 -33.3% 5311.54: server (16 days) (lookup=false) 0.02 0.03 +50.0% 5311.55: size (16 days) 37.3K 186.0K +398.2% 5311.56: client (16 days) (lookup=false) 0.03 0.02 -33.3% 5311.58: server (32 days) (lookup=false) 0.03 0.03 +0.0% 5311.59: size (32 days) 46.5K 197.2K +324.3% 5311.60: client (32 days) (lookup=false) 0.03 0.02 -33.3% 5311.62: server (64 days) (lookup=false) 0.25 0.17 -32.0% 5311.63: size (64 days) 1.5M 5.1M +239.8% 5311.64: client (64 days) (lookup=false) 0.43 0.37 -14.0% 5311.66: server (128 days) (lookup=false) 0.50 0.29 -42.0% 5311.67: size (128 days) 4.1M 9.8M +138.6% 5311.68: client (128 days) (lookup=false) 0.87 0.67 -23.0%
It's important to realize that even with the test variable, the path-walk logic is overriding the bitmap logic in the HEAD~3 case.
What's happening is that the path-walk mode (without bitmaps) is computing a smaller packfile for all of these cases. Some are significantly smaller, but only when it's a very small pack anyway. The bitmap case is faster only for larger fetches.
I did the same test without the path-walk feature and both columns looked the same (as expected, no change due to this series) and the data matched the path-walk test's HEAD column pretty closely. So this shows that adding path-walk to bitmap-focused efforts is not a regression on any of these dimensions.
This test was for my local copy of the Git repository, including all the forks I fetch. I hoped the results would be different for repositories that have data shapes that struggle with name-hash collisions, but microsoft/fluentui is an example that I've used for path-walk repacks before and it had similar data.
Do you have a good feeling for why the path-walk feature doesn't make a huge change in these test scenarios?
Thanks, -Stolee