Re: [PATCH 2/2] builtin/repo: don't fully parse trees
- From
Kristofer Karlsson <krka@spotify.com>
- Date
- Nov 13, 2026, 14:15 UTC
- Message-ID
- <CAL71e4PscLozeuQbwNePkgn=QdDPhE1zqTtuGahnKWDN++B0Hw@mail.gmail.com>
- In-Reply-To
- <20261113-pks-repo-count-objects-speedup-v1-2-7ba4f29e74fb@pks.im>
On Fri, 13 Nov 2026 at 14:06, Patrick Steinhardt <ps@pks.im> wrote:
Show 10 quoted lines
> > Benchmark 1: ./git repo structure (commit = HEAD~) (10 runs) > mean ± σ min … max > Wall Time 9.635 s ± 0.072 s 9.534 s … 9.799 s > Memory 3.4 GiB ± 0.0 GiB 3.4 GiB … 3.4 GiB > > Benchmark 2: ./git repo structure (commit = HEAD) (10 runs) > mean ± σ min … max > Wall Time 7.676 s ± 0.106 s 7.568 s … 7.823 s -20.3% > Memory 0.7 GiB ± 0.0 GiB 0.7 GiB … 0.7 GiB -80.4% (5.1x less)
That's quite a nice memory reduction!
> But besides the speedup, the bigger win here is that we no longer keep > these parsed trees in memory at all, which translates in a significant > reduction in peak memory usage.
I took a look at this since it is very similar to what I recently did for the incremental connectivity check, where the tree verifier also reads raw tree buffers instead of going through parse_object_buffer(), for the same memory reasons (I only needed to track the oids, not the full objects).
Both patches look good to me.
Thanks, Kristofer