Re: Slow git pack-refs --all

From: Jeff King <peff@peff.net>
Date: Jan 2, 2026, 07:49 UTC
Message-ID: <20260102074901.GD2581074@coredump.intra.peff.net>
In-Reply-To: <CH3PR12MB9026DFCF7AF4ED1A249B16A5C2BDA@CH3PR12MB9026.namprd12.prod.outlook.com>

On Wed, Dec 31, 2025 at 05:48:11AM +0000, Martin Fick wrote:

Show 18 quoted lines

> >> I think this is from `should_pack_ref`:
> >>
> >>     /* Do not pack broken refs: */
> >>     if (!ref_resolves_to_object(ref->name, refs->base.repo, ref->oid, ref->flags))
> >>       return 0;
> >>
> >> So Git is going to need to verify that the object at least exists.  I
> >> don't know why we would need to _open_ them, however.  Perhaps someone
> >> else has ideas.
> >
> >The packed-refs file stores tag-peeling information. So pack-refs opens
> >the object for any newly written ref via peel_object(), which has to at
> >least read the header to get the type. That call happens via
> >write_with_updates() in packed-backend.c.
> 
> Thanks, this makes sense. However, since jgit needs to peel these
> objects also, it doesn't make sense to me that this would be the
> bottleneck unless git is doing something terribly inefficient here. :(

I'd expect both git and jgit to open each loose object once. I tried running "git pack-refs --all --prune" under strace on a test repo. It does seem to open the object once. Then I tried the same with jgit (though it does not understand --prune), and got the same results.

So...I dunno.

Show 5 quoted lines

> Except for the fact that repacking objects made it faster, my
> observations make it look like it's the writing that is actually slow,
> not the reads. Could there be too many small unbuffered writes, could
> this write path have missed being optimized (it likely isn't used
> elsewhere)?

All of the packed-refs writes are through fprintf(), which should be fully buffered. You should be able to confirm with strace (I get 4096-byte writes on my system).

If writing were slow, I'd also expect that to scale with the total number of refs, not the number of changed refs (since we have to rewrite the whole file, but only new entries need to be peeled).

-Peff