Re: [BUG] "git describe --match" performance

From: Jeff King <peff@peff.net>
Date: Oct 31, 2024, 17:20 UTC
Message-ID: <20241031172021.GB2277590@coredump.intra.peff.net>
In-Reply-To: <469d82f8-4883-4c87-b681-288a0520e745@app.fastmail.com>

On Thu, Oct 31, 2024 at 05:14:01PM +0100, Kristoffer Haugsbakk wrote:

Show 17 quoted lines

> On Thu, Oct 31, 2024, at 12:47, Jeff King wrote:
> >   git commit-graph write --reachable
> >
> > That reduces the slow case for me by a factor of 10. And likewise other
> > traversal operations should get faster.
> >
> > I think we'll build the commit graph file by default these days when you
> > run "git gc". But we don't build it immediately after cloning. Perhaps
> > we should change that.
> >
> > -Peff
> 
> There was this thread from last year where Rohit cloned Linux and the
> command took more than twelve seconds.  Then git-commit-graph(1) fixed
> it.
> 
> https://lore.kernel.org/git/CAKazavxTXwcZFtL2XyU3MpaUR=snWY8w8Lwpco+mkbqm2nWE=w@mail.gmail.com/

Yeah, there's discussion in the linked thread there about running "git gc" after clone. If "gc --auto" was pruned down to run some minimal bits, that might be OK. What we definitely _don't_ want to do is run "git repack", because it's very expensive to do the full object graph walk (and buys nothing on a freshly cloned repo unless you do the even more expensive "-f").

So the simplest path forward, but which is a little messy, is to just run "commit-graph write" after the clone. In fact, we already have fetch.writeCommitGraph (though it does still default to "false"). I'd expect that to work with clone (since it's conceptually init+fetch), but it doesn't seem to. Looks like the code to trigger it is directly in builtin/fetch.c, and clone triggers the fetch itself internally.

But I don't think factoring that out and calling it from both places would be too hard.

It's a bigger question whether people might be annoyed by some extra computation at clone time. But I suspect it's OK in practice. Even ignoring the cost of moving it over the network, my 8-core machine takes almost 3 minutes to index the linux.git packfile on clone. Building the commit graph after that takes about 13 seconds.

-Peff