git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Slow git pack-refs --all

From
brian m. carlson <sandals@crustytoothpaste.net>
Date
Dec 26, 2025, 17:15 UTC
Message-ID
<aU7Cs2pXiXInfBh4@fruit.crustytoothpaste.net>
In-Reply-To
<20251226044507.GA1971832@coredump.intra.peff.net>
On 2025-12-26 at 04:45:07, Jeff King wrote:
Show 22 quoted lines
> On Thu, Dec 25, 2025 at 11:38:30PM +0000, brian m. carlson wrote:
> 
> > I think this is from `should_pack_ref`:
> > 
> >     /* Do not pack broken refs: */
> >     if (!ref_resolves_to_object(ref->name, refs->base.repo, ref->oid, ref->flags))
> >     	return 0;
> > 
> > So Git is going to need to verify that the object at least exists.  I
> > don't know why we would need to _open_ them, however.  Perhaps someone
> > else has ideas.
> 
> The packed-refs file stores tag-peeling information. So pack-refs opens
> the object for any newly written ref via peel_object(), which has to at
> least read the header to get the type. That call happens via
> write_with_updates() in packed-backend.c.
> 
>   If we wanted to be really pedantic, anything in refs/heads/ should not
>   point to a non-commit and thus should never need to be peeled. I'm not
>   sure if we want to embed that assumption in this code path, though
>   (nor would it necessarily help Martin's case if the refs are not in
>   refs/heads anyway).

I don't think that would be a good idea. I know that people definitely do updates of the loose refs by hand (although they should not) and so it's entirely possible for them to contain invalid values, such as having branches contain non-commit objects.

I wonder if reftable would avoid the need for this kind of expensive check since it would already have the data peeled if need be and wouldn't need to recompute the values.

-- 
brian m. carlson (they/them)
Toronto, Ontario, CA
Previous: Jeff KingNext: Jeff King
Message 4 of 19 in “Slow git pack-refs --all”
  1. Martin FickDec 25, 2025
  2. brian m. carlsonDec 25, 2025
  3. Jeff KingDec 26, 2025
  4. brian m. carlsonDec 26, 2025
  5. Jeff KingDec 27, 2025
  6. Martin FickDec 31, 2025
  7. Jeff KingJan 2, 2026
  8. Martin FickJan 5, 2026
  9. Patrick SteinhardtJan 6, 2026
  10. Martin FickJan 6, 2026
  11. Patrick SteinhardtJan 7, 2026
  12. Martin FickJan 7, 2026
  13. Patrick SteinhardtJan 8, 2026
  14. Jeff KingJan 15, 2026
  15. Martin FickJan 16, 2026
  16. Martin FickJan 7, 2026
  17. Jeff KingJan 6, 2026
  18. Martin FickJan 6, 2026
  19. Martin FickDec 31, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.