Re: git-diff in a worktree is an order of magnitude slower?
- From
Jeff King <peff@peff.net>
- Date
- Jun 21, 2026, 17:45 UTC
- Message-ID
- <20260621174518.GB2206349@coredump.intra.peff.net>
- In-Reply-To
- <20260621172432.GA2206349@coredump.intra.peff.net>
On Sun, Jun 21, 2026 at 01:24:32PM -0400, Jeff King wrote:
Show 14 quoted lines
> I think this is the core of the issue. These entries are "racy git > dirty" in the sense that their mtimes are the same as the index mtime, > and so we double-check the contents. This is the first bullet point > under the "Racy Git" section of Documentation/technical/racy-git.adoc. > > But diffcore_skip_stat_unmatch() doesn't count them as dirty, so we > don't increment the counter, and thus top-level git-diff won't write out > the new index. And thus every subsequent diff repeats the same > expensive double-check. > > But I'm not sure where the blame lies. Either: > > 1. diffcore_skip_stat_unmatch() should be counting these in its > "dirty" counter; or
BTW, I don't think diffcore actually has the information it would need to do so. The racy stuff is handled under the hood in ie_match_stat(), which returns only a set of "changed" flags. So the caller cannot tell the difference between the two cases:
1. We checked ce_match_stat_basic() which said "no change", and then
is_racy_timestamp() was false, so that was good enough. 2. is_racy_timestamp() is true, so we further did a content check,
found nothing, and returned the same "no change"Obviously we could pass back another flag, but that would disrupt the other callers. Hmm. It looks like we could pass in a flag to say "assume racy entries are modified". And then they come back to the diff code, diffcore_skip_stat_unmatch() sees they're not real diffs and suppresses them, but we _do_ count them as stat-dirty.
Like this:
diff --git a/builtin/diff.c b/builtin/diff.c index 4b46e394ce..4d36b5c1e0 100644 --- a/builtin/diff.c +++ b/builtin/diff.c @@ -271,6 +271,9 @@ static void builtin_diff_files(struct rev_info *revs, int argc, const char **arg argv++; argc--; } + if (revs->diffopt.skip_stat_unmatch) + options |= DIFF_RACY_IS_MODIFIED; + /* * "diff --base" should not combine merges because it was not * asked to. "diff -c" should not densify (if the user wants That seems to work, in the sense that "git diff" does refresh the index afterwards. But the timings are a bit funny. In my working tree of linux.git with many racy entries it was ~500ms to do the first diff (and the second, and so on, because we never updated the index). After the patch above, it is 1800ms to do the first diff, and then fast (~30ms) after. I could believe it takes twice as long when we refresh the index (because I don't think we use the stat-cleanliness we collected from the diff, but rather just do a from-scratch index refresh). But that would imply it should take ~1000ms. Where does the extra 800ms go? I guess that somehow the content-check done by diffcore_skip_stat_unmatch() is slower than the one done by ie_match_stat(). I think the individual functions are respectively diff_filespec_check_stat_unmatch() and ce_modified_check_fs(). I don't know if any of this is really worth digging too far. This feels like a case we could do a bit better at, but I wonder how much it matters in practice. As soon as you do any index-refresh (including "git status"), the racy entries are cleared and everything is faster. It just seems kind of lame that we write out the initial working tree with so many racy entries. -Peff