{"thread":{"id":"63694","subject":"[perf] git log --follow seems slow","startedAt":"2025-06-25T16:44:29Z","lastAt":"2025-06-26T18:52:18Z","messageCount":2,"participants":["Kai Koponen","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"520721","messageId":"CADYQcGoAN8bW3SVEdqPLJAMQbicbo6wQW_nNa_uYkXMouxO=Ng@mail.gmail.com","threadId":"63694","inReplyTo":null,"subject":"[perf] git log --follow seems slow","fromName":"Kai Koponen","fromEmail":"kaikoponen@google.com","sentAt":"2025-06-25T16:44:16Z","receivedAt":"2025-06-25T16:44:29Z","isPatch":false,"sender":{"key":"kaikoponen@google.com","avatar":"https://avatars.githubusercontent.com/u/161378660?v=4"},"body":"Sample case:\n```\ngit clone https://github.com/golang/go.git\ncd go\ngit config core.commitGraph true\ngit commit-graph write --split --reachable --changed-paths\ngit checkout 3730814f2f2bf24550920c39a16841583de2dac1  # Reproduce HEAD\n\n# Observe that there are no commits to this file until 53fd522\ntime git rev-list -1 HEAD -- test/index0.go # 0.07s\n\n# Check size of test/index0.go and of 53fd522's diff\nwc -l test/index0.go # 12\ngit show --stat 53fd522c0db58f3bd75d85295f46bb06e8ab1a9b --\ntest/index0.go # 1 line modified\n\n# Find commit 53fd522 with --follow enabled\ntime git log --pretty=format:%H --follow --max-count=1 HEAD --\ntest/index0.go # 2.3s (!)\n```\n\nAs far as I can tell, even with all of git's expensive rename and copy\nheuristics, the above call shouldn't take this long (nor should higher\nmax-counts, but count 1 is easier to ballpark the work required):\n- There are no other commits touching this file after the target commit.\n- Even if some commit after the target commit copied index0.go to\nanother file, such a commit\n  would not need to be included in index0.go's log, so there should be no need\n  to do any sort of copy-checking work until index0.go is touched.\n- The target commit is not the creation of the file nor is it a > 50%\nrewrite of the file, so it should\n  not be possible for it to count as a rename or copy, and therefore\nno other file blobs should\n  need to be checked to confirm this (also, we only requested one\ncommit, so even if the target\n  commit were a rename, in theory there would be no need to check this).\n\nI assume one or more of the following are going on:\n- I'm missing some nuance / reason why rename/copy detection is\nnecessarily more expensive than I think, even for commits that don't\ntouch the file\n- --follow is not taking advantage of the commit graph (seems likely\nas --follow seems equally slow with core.commitGraph off)\n- --follow is performing expensive rename/copy checks even on small\ndiffs that are not file creations\n- --follow is not using the --max-count target to limit its work\n- --follow is not using the --format output to limit its work? (Are\nthere format options that require more work?)\n\nGit version: 2.50.0.714.g196bf9f422-goog\n"},{"id":"520780","messageId":"20250626185216.GB17172@coredump.intra.peff.net","threadId":"63694","inReplyTo":"CADYQcGoAN8bW3SVEdqPLJAMQbicbo6wQW_nNa_uYkXMouxO=Ng@mail.gmail.com","subject":"Re: [perf] git log --follow seems slow","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2025-06-26T18:52:16Z","receivedAt":"2025-06-26T18:52:18Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Jun 25, 2025 at 12:44:16PM -0400, Kai Koponen wrote:\n\n> - --follow is not taking advantage of the commit graph (seems likely\n> as --follow seems equally slow with core.commitGraph off)\n\nIt's mostly this. History pruning is disabled with --follow, since the\npath of interest may change as we traverse through history. And the use\nof commit graph changed-path bloom filters is tied to the pruning code.\nSo this hits you two-fold:\n\n  1. For each commit we visit, we actually do a tree diff to see if it\n     touched that path (leaving aside rename detection, we still have to\n     open the trees down to the path of interest).\n\n  2. We don't prune side-branches of history, so we are visiting more\n     commits. Normally when we see a merge of two branches, when one\n     parent matches the merge result and the other does not, we know\n     that nothing interesting happened on the side branch. So we do not\n     traverse those commits at all. But with pruning disabled, that no\n     longer happens.\n\nThese limitations are mostly due to the hacky way that --follow is\nbolted on to the traversal machinery. The \"right\" way for --follow to\nwork is to keep a unique pathspec (and bloom filter) for each line of\ncommits, passing it down as we walk backwards through history (and\nmodifying it when we see a rename).\n\nIt's probably possible to bolt bloom-filter checks onto the existing\n--follow feature, but it would probably involve a lot of special casing.\n\n-Peff\n"}]}