git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: git-diff in a worktree is an order of magnitude slower?

From
Jeff King <peff@peff.net>
Date
Jun 21, 2026, 17:45 UTC
Message-ID
<20260621174518.GB2206349@coredump.intra.peff.net>
In-Reply-To
<20260621172432.GA2206349@coredump.intra.peff.net>
On Sun, Jun 21, 2026 at 01:24:32PM -0400, Jeff King wrote:
Show 14 quoted lines
> I think this is the core of the issue. These entries are "racy git
> dirty" in the sense that their mtimes are the same as the index mtime,
> and so we double-check the contents. This is the first bullet point
> under the "Racy Git" section of Documentation/technical/racy-git.adoc.
> 
> But diffcore_skip_stat_unmatch() doesn't count them as dirty, so we
> don't increment the counter, and thus top-level git-diff won't write out
> the new index. And thus every subsequent diff repeats the same
> expensive double-check.
> 
> But I'm not sure where the blame lies. Either:
> 
>   1. diffcore_skip_stat_unmatch() should be counting these in its
>      "dirty" counter; or

BTW, I don't think diffcore actually has the information it would need to do so. The racy stuff is handled under the hood in ie_match_stat(), which returns only a set of "changed" flags. So the caller cannot tell the difference between the two cases:

  1. We checked ce_match_stat_basic() which said "no change", and then
     is_racy_timestamp() was false, so that was good enough.
  2. is_racy_timestamp() is true, so we further did a content check,
     found nothing, and returned the same "no change"

Obviously we could pass back another flag, but that would disrupt the other callers. Hmm. It looks like we could pass in a flag to say "assume racy entries are modified". And then they come back to the diff code, diffcore_skip_stat_unmatch() sees they're not real diffs and suppresses them, but we _do_ count them as stat-dirty.

Like this:
diff --git a/builtin/diff.c b/builtin/diff.c
index 4b46e394ce..4d36b5c1e0 100644
--- a/builtin/diff.c
+++ b/builtin/diff.c
@@ -271,6 +271,9 @@ static void builtin_diff_files(struct rev_info *revs, int argc, const char **arg
 		argv++; argc--;
 	}
 
+	if (revs->diffopt.skip_stat_unmatch)
+		options |= DIFF_RACY_IS_MODIFIED;
+
 	/*
 	 * "diff --base" should not combine merges because it was not
 	 * asked to.  "diff -c" should not densify (if the user wants

That seems to work, in the sense that "git diff" does refresh the index
afterwards. But the timings are a bit funny.

In my working tree of linux.git with many racy entries it was ~500ms to
do the first diff (and the second, and so on, because we never updated
the index). After the patch above, it is 1800ms to do the first diff,
and then fast (~30ms) after.

I could believe it takes twice as long when we refresh the index
(because I don't think we use the stat-cleanliness we collected from the
diff, but rather just do a from-scratch index refresh). But that would
imply it should take ~1000ms. Where does the extra 800ms go? I guess
that somehow the content-check done by diffcore_skip_stat_unmatch() is
slower than the one done by ie_match_stat(). I think the individual
functions are respectively diff_filespec_check_stat_unmatch() and
ce_modified_check_fs().

I don't know if any of this is really worth digging too far. This feels
like a case we could do a bit better at, but I wonder how much it
matters in practice. As soon as you do any index-refresh (including "git
status"), the racy entries are cleared and everything is faster. It
just seems kind of lame that we write out the initial working tree with
so many racy entries.

-Peff
Previous: Jeff KingNext: Junio C Hamano
Message 11 of 19 in “git-diff in a worktree is an order of magnitude slower?”
  1. D. Ben KnobleJun 8, 2026
  2. Jeff KingJun 9, 2026
  3. D. Ben KnobleJun 9, 2026
  4. Jeff KingJun 11, 2026
  5. Junio C HamanoJun 11, 2026
  6. brian m. carlsonJun 11, 2026
  7. D. Ben KnobleJun 20, 2026
  8. Junio C HamanoJun 21, 2026
  9. Junio C HamanoJun 21, 2026
  10. Jeff KingJun 21, 2026
  11. Jeff KingJun 21, 2026
  12. Junio C HamanoJun 21, 2026
  13. Jeff KingJun 21, 2026
  14. Junio C HamanoJun 21, 2026
  15. Junio C HamanoJun 22, 2026
  16. Jeff KingJun 28, 2026
  17. Junio C HamanoJun 21, 2026
  18. Jeff KingJun 21, 2026
  19. D. Ben KnobleJul 3, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.