Re: [PATCH] packfile: fix perf regression with many packs
- From
Patrick Steinhardt <ps@pks.im>
- Date
- Aug 13, 2026, 07:35 UTC
- Message-ID
- <an1z3uy7Xtqw3U_l@pks.im>
- In-Reply-To
- <20260812212955.GA152730@coredump.intra.peff.net>
On Wed, Aug 12, 2026 at 05:29:55PM -0400, Jeff King wrote:
Show 31 quoted lines
> On Wed, Aug 12, 2026 at 12:51:30PM -0700, Junio C Hamano wrote: > > > "Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com> > > writes: > > > > > In one reported use case (https://github.com/microsoft/git/issues/970), > > > N equals 37,815 and caused a slow-down of a simple `git rev-parse > > > --short HEAD` (which is regularly executed as part of `GIT_PS1`) from > > > 0.4s to 4.5s. In another, heavily exercised CI scenario, clone times > > > increased from under 2 minutes to over half an hour. > > > > Face with Rolling Eyes (1f644) 🙄 > > > > As we grow older, more and more extreme use cases that we initially > > thought were simply crazy become reality. > > Sort of. The quadratic adding became a problem long ago, hence > ec48540fe8 (packfile.c: speed up loading lots of packfiles, 2019-11-27). > > So this was something we already dealt with that regressed. We can even > see the regression in our perf suite: > > $ GIT_SKIP_TESTS='p5303.[1-9] p5303.1[0-9]' ./run 589127caa730^ 589127caa730 p5303-many-packs.sh > Test 589127caa730^ 589127caa730 > ---------------------------------------------------------------------- > 5303.21: load 10,000 packs 0.13(0.11+0.02) 0.45(0.42+0.02) +246.2% > > Unfortunately I don't think anybody pays close attention to the perf > suite (partially because it's clunky and expensive to run, but also > because it often requires human judgement to decide when something is a > real change and not just a blip).
Yeah, that's a problem indeed. At GitLab we do have Bencher set up for continuous benchmarking [1], but due to recent changes to our CI setup those are now very flaky because seemingly, we flip-flop between two different runners that have different specs. But we're obviously missing a test there with lots of packfiles, so we didn't catch this regression.
Thanks!
Patrick
[1]: https://bencher.dev/perf/git/plots