[PATCH v2 0/2] builtin/repo: improve counting of objects
- From
Patrick Steinhardt <ps@pks.im>
- Date
- Nov 16, 2026, 08:01 UTC
- Message-ID
- <20261116-pks-repo-count-objects-speedup-v2-0-9c8be218eaff@pks.im>
- In-Reply-To
- <20261113-pks-repo-count-objects-speedup-v1-0-7ba4f29e74fb@pks.im>
Hi,
I was recently using `git repo structure` in a very large repository and was eventually greeted with an OOM. So I had a bit of a look and found a way to improve the memory footprint and speed it up a bit.
The repository still doesn't produce results. As it turns out, walking 33 billion tree entries takes a while, and everything rev-list based that walks trees is kinda broken in there. Oh, well, a problem for another day.
Changes in v2:
- Reword a comment to point out that parsing the commit should in
theory not be required.
- Link to v1: https://patch.msgid.link/20261113-pks-repo-count-objects-speedup-v1-0-7ba4f29e74fb@pks.imThanks!
Patrick
---
Patrick Steinhardt (2):
builtin/repo: only parse objects as needed
builtin/repo: don't fully parse treesbuiltin/repo.c | 34 +++++++++++++++++++--------------- 1 file changed, 19 insertions(+), 15 deletions(-)
Range-diff versus v1:
1: a7a1c1a8041 ! 1: 2780e491ead builtin/repo: only parse objects as needed
@@ builtin/repo.c: static int count_objects(const char *path UNUSED, struct oid_arr
case OBJ_COMMIT:
- commit = object_as_type(obj, OBJ_COMMIT, 0);
+ /*
-+ * Commits have already been parsed by the revision
-+ * walk, so there is no need to read their contents
-+ * again.
++ * Commits should already have been parsed by the
++ * revision walk, so in theory we do not need to read
++ * their contents again.
+ */
+ commit = lookup_commit(data->odb->repo, &oids->oid[i]);
+ if (!commit || repo_parse_commit(data->odb->repo, commit))
2: d9fd2366b84 = 2: aa32dcb2961 builtin/repo: don't fully parse trees--- base-commit: 1891de46f79dc444a47db1c87bd0c8c76b5463f2 change-id: 20261111-pks-repo-count-objects-speedup-528abf12ea23