Re: [PATCH 3/6] object-file: extract logic to approximate object count
- From
Patrick Steinhardt <ps@pks.im>
- Date
- Mar 11, 2026, 13:58 UTC
- Message-ID
- <abF084pB38G5Nyv6@pks.im>
- In-Reply-To
- <87v7f2lei6.fsf@iotcl.com>
On Wed, Mar 11, 2026 at 01:47:13PM +0100, Toon Claes wrote:
Show 9 quoted lines
> Patrick Steinhardt <ps@pks.im> writes: > > > In "builtin/gc.c" we have some logic that checks whether we need to > > repack objects. This is done by counting the number of objects that we > > have and checking whether it exceeds a certain threshold. We don't > > really need an accurate object count though, which is why we only > > open a single object diretcroy shard and then extrapolate from there. > > s/diretcroy/directory/
Thanks, fixed locally.
Show 45 quoted lines
> > diff --git a/object-file.c b/object-file.c
> > index a3ff7f586c..da67e3c9ff 100644
> > --- a/object-file.c
> > +++ b/object-file.c
> > @@ -1868,6 +1868,47 @@ int odb_source_loose_for_each_object(struct odb_source *source,
> > NULL, NULL, &data);
> > }
> >
> > +int odb_source_loose_approximate_object_count(struct odb_source *source,
> > + unsigned long *out)
> > +{
> > + const unsigned hexsz = source->odb->repo->hash_algo->hexsz - 2;
> > + unsigned long count = 0;
> > + struct dirent *ent;
> > + char *path = NULL;
> > + DIR *dir = NULL;
> > + int ret;
> > +
> > + path = xstrfmt("%s/17", source->path);
> > +
> > + dir = opendir(path);
> > + if (!dir) {
> > + if (errno == ENOENT) {
> > + *out = 0;
> > + ret = 0;
> > + goto out;
> > + }
> > +
> > + ret = error_errno("cannot open object shard '%s'", path);
> > + goto out;
> > + }
> > +
> > + while ((ent = readdir(dir)) != NULL) {
> > + if (strspn(ent->d_name, "0123456789abcdef") != hexsz ||
> > + ent->d_name[hexsz] != '\0')
> > + continue;
> > + count++;
> > + }
> > +
> > + *out = count * 256;
>
> This makes the number way larger, but I don't think we need to worry
> getting anywhere near ULONG_MAX, because I would expect to have Git
> coming to a grind way before that happens (not to mention filesystems
> would get unhappy about it too).Yup. Even if `unsigned long` was 32 bits that would be >128 million loose objects in a single directory. I agree that this is probably going to make some things in Git unhappy. So we could have overflow checks here, but I'm not sure it's worth it.
Thanks!
Patrick