Re: git-diff in a worktree is an order of magnitude slower?
- From
D. Ben Knoble <ben.knoble@gmail.com>
- Date
- Jun 9, 2026, 17:15 UTC
- Message-ID
- <CALnO6CD+3sE1xQUnRsCFfWrZTsq2Edw7BWseLzasgT3dgtaq_Q@mail.gmail.com>
- In-Reply-To
- <20260609001134.GD358144@coredump.intra.peff.net>
On Mon, Jun 8, 2026 at 8:11 PM Jeff King <peff@peff.net> wrote:
Show 43 quoted lines
>
> On Mon, Jun 08, 2026 at 07:36:45PM -0400, D. Ben Knoble wrote:
>
> > I'd like to report and offer to help fix what I view as a serious performance
> > bug:
> >
> > "git diff --no-ext-diff --quiet" performs about ~10x slower in a secondary
> > worktree than in the main worktree.
>
> Hmm, I get the opposite effect: it is much faster in the worktree!
>
> I did:
>
> git clone /path/to/linux.git
> git -C linux worktree add --detach ../wt
> hyperfine -L dir linux,wt 'git -C {dir} diff'
>
> which yielded:
>
> Benchmark 1: git -C linux diff
> Time (mean ± σ): 188.9 ms ± 2.5 ms [User: 166.4 ms, System: 130.7 ms]
> Range (min … max): 185.5 ms … 194.8 ms 16 runs
>
> Benchmark 2: git -C wt diff
> Time (mean ± σ): 20.0 ms ± 1.5 ms [User: 23.4 ms, System: 103.5 ms]
> Range (min … max): 17.2 ms … 24.6 ms 132 runs
>
> Summary
> git -C wt diff ran
> 9.43 ± 0.71 times faster than git -C linux diff
>
> Running:
>
> perf record -g git -C wt --no-pager diff
> perf record -g git -C linux --no-pager diff
> perf diff
>
> implies that the slow case is spending a lot more time computing sha1s.
> Which implies that the entries are stat dirty. And indeed, if I run:
>
> git -C linux update-index --refresh
>
> now they both take ~20ms.Ah, TIL about --refresh. I suppose it could be nice if "git diff" updated the index in this way, but that sounds like a band-aid. Maybe creating a fresh worktree should do the equivalent to make sure it's considered "fresh"?
(This also dropped my timings down to normal.)
At $DAYJOB, I _think_ some version of "git restore <stuff>" ended up also updating the index.
Show 9 quoted lines
> I wonder if it's just a racy-git problem? Many files are written in the > same second as the index, so they end up with the same mtimes, and we > have to err on the side of checking the contents. > > See Documentation/technical/racy-git.adoc for a larger discussion. > > So it is not really about worktrees at all, but just "bad luck" in > generating that initial index (that goes away next time you actually > make an index update that rewrites the whole thing).
Ah, that makes sense! I'm familiar with the raciness but didn't expect it here.
> I'd have thought USE_NSEC was the default these days, but looks like it > isn't? Try building with that and I'll bet it goes away entirely.
Thanks, I'll take a look.
I can see on my Macbook that at least Meson does automatically set either USE_ST_TIMESPEC or NO_NSEC automatically, but has no option to enabled USE_NSEC and try that. I can probably write that patch (which I'll do to test), and I can send it along with the "worktree add should refresh the index" if you think that's an appropriate thing to do.
Show 8 quoted lines
> > PS I almost CC'd Peff and Patrick, whose names stood out in "git
> > shortlog builtin/{worktree,diff}* object-file* | sort -t\( -k2 -g",
> > but decided they'd be their own best judge of whether they can
> > understand what's going on? :)
>
> You might be interested in "git shortlog -ns". :)
>
> -PeffPhew! Yeah, that's much nicer. Thanks! (When typing this out for email, I even left out the "grep '^[^[:space:]]' |" filter :P)
-- D. Ben Knoble