{"thread":{"id":"66873","subject":"[PATCH] diff-lib: skip leading path check for fsmonitor-valid entries","startedAt":"2026-11-18T18:22:47Z","lastAt":"2026-11-19T22:06:33Z","messageCount":4,"participants":["Maik Riechert via GitGitGadget","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"558732","messageId":"pull.2318.git.1795026168007.gitgitgadget@gmail.com","threadId":"66873","inReplyTo":null,"subject":"[PATCH] diff-lib: skip leading path check for fsmonitor-valid entries","fromName":"Maik Riechert via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-11-18T18:22:47Z","receivedAt":"2026-11-18T18:22:47Z","isPatch":true,"sender":{"key":"name:Maik Riechert","avatar":null},"body":"From: Maik Riechert <letmaik@outlook.com>\n\nSince 4f3d6d0261 (fsmonitor: skip lstat deletion check during git\ndiff-index, 2021-03-17), check_removed() does not lstat() entries\nthat fsmonitor says are unchanged. But it still calls\nhas_symlink_leading_path() for them, which lstat()s the directories\nleading up to each path.\n\nThat check is there to notice a leading directory that was replaced\nby a symlink. fsmonitor would have reported such a change, and\nrefresh_fsmonitor() would have cleared CE_FSMONITOR_VALID for every\nentry below it, so we would not get here. preload_index() already\ntrusts the flag this way and skips these entries altogether.\n\nThe S_ISDIR() check that follows cannot trigger either: the mode\ncomes from the index, and gitlinks are never marked valid.\n\nSo return as soon as fake_lstat() has filled in the stat data.\n\nThe first new test checks the shortcut itself by replacing a parent\ndirectory with a symlink without telling fsmonitor. The other two make\nsure we still notice when fsmonitor does report that directory, with\nand without a trailing slash, since nothing else checks for it now.\n\nOn Linux, with 100,000 files (ten per directory), a hook that\nreports nothing and a warm cache, \"git diff HEAD\" goes from 57ms to\n26ms.\n\nSigned-off-by: Maik Riechert <letmaik@outlook.com>\n---\n    diff-lib: skip leading path check for fsmonitor-valid entries\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-2318%2Fletmaik%2Fdiff-lib-fsmonitor-leading-path-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2318/letmaik/diff-lib-fsmonitor-leading-path-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/2318\n\n diff-lib.c                  |  8 +++----\n t/t7519-status-fsmonitor.sh | 44 +++++++++++++++++++++++++++++++++++++\n 2 files changed, 48 insertions(+), 4 deletions(-)\n\ndiff --git a/diff-lib.c b/diff-lib.c\nindex 086476bd77..3730e04149 100644\n--- a/diff-lib.c\n+++ b/diff-lib.c\n@@ -43,10 +43,10 @@ static int check_removed(const struct cache_entry *ce, struct stat *st)\n {\n \tint stat_err;\n \n-\tif (!(ce->ce_flags & CE_FSMONITOR_VALID))\n-\t\tstat_err = lstat(ce->name, st);\n-\telse\n-\t\tstat_err = fake_lstat(ce, st);\n+\tif (ce->ce_flags & CE_FSMONITOR_VALID)\n+\t\treturn fake_lstat(ce, st);\n+\n+\tstat_err = lstat(ce->name, st);\n \tif (stat_err < 0) {\n \t\tif (!is_missing_file_error(errno))\n \t\t\treturn -1;\ndiff --git a/t/t7519-status-fsmonitor.sh b/t/t7519-status-fsmonitor.sh\nindex 93973ed25a..6c9df9f333 100755\n--- a/t/t7519-status-fsmonitor.sh\n+++ b/t/t7519-status-fsmonitor.sh\n@@ -336,6 +336,50 @@ do\n \tdone\n done\n \n+# Create a repository whose only entry, dir1/tracked, is marked valid\n+# by an fsmonitor hook that reports no changes, then replace dir1 with\n+# a symlink to a directory that has the same contents.\n+setup_symlinked_leading_dir () {\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tmkdir repo/dir1 &&\n+\t: >repo/dir1/tracked &&\n+\tgit -C repo add dir1/tracked &&\n+\tgit -C repo commit -m initial &&\n+\ttest_hook -C repo --setup fsmonitor-test <<-\\EOF &&\n+\tprintf \"last_update_token\\0\"\n+\tEOF\n+\tgit -C repo config core.fsmonitor .git/hooks/fsmonitor-test &&\n+\tgit -C repo update-index --refresh --force-write-index &&\n+\techo \"h dir1/tracked\" >expect &&\n+\tgit -C repo ls-files -f >actual &&\n+\ttest_cmp expect actual &&\n+\tmv repo/dir1 repo/dir2 &&\n+\tln -s dir2 repo/dir1\n+}\n+\n+# Like \"status\" above, diff-index should trust fsmonitor and not look\n+# for symlinks in leading directories that it was not told about.\n+test_expect_success SYMLINKS \"diff-index doesn't detect unreported symlinked directories\" '\n+\tsetup_symlinked_leading_dir &&\n+\tgit -C repo diff-index --name-status HEAD >actual &&\n+\ttest_must_be_empty actual\n+'\n+\n+for report in dir1 dir1/\n+do\n+\ttest_expect_success SYMLINKS \"diff-index detects symlinked directory reported as $report\" '\n+\t\tsetup_symlinked_leading_dir &&\n+\t\ttest_hook -C repo --clobber fsmonitor-test <<-EOF &&\n+\t\tprintf \"last_update_token\\0\"\n+\t\tprintf \"$report\\0\"\n+\t\tEOF\n+\t\tprintf \"D\\tdir1/tracked\\n\" >expect &&\n+\t\tgit -C repo diff-index --name-status HEAD >actual &&\n+\t\ttest_cmp expect actual\n+\t'\n+done\n+\n # test that splitting the index doesn't interfere\n test_expect_success 'splitting the index results in the same state' '\n \twrite_integration_script &&\n\nbase-commit: e1ec09b23c8e0deb21a0b95ac5996422277395d4\n-- \ngitgitgadget\n\n"},{"id":"558737","messageId":"xmqqse0y55gp.fsf@gitster.g","threadId":"66873","inReplyTo":"pull.2318.git.1795026168007.gitgitgadget@gmail.com","subject":"Re: [PATCH] diff-lib: skip leading path check for fsmonitor-valid entries","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-11-18T19:01:42Z","receivedAt":"2026-11-18T19:01:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Maik Riechert via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> diff --git a/diff-lib.c b/diff-lib.c\n> index 086476bd77..3730e04149 100644\n> --- a/diff-lib.c\n> +++ b/diff-lib.c\n> @@ -43,10 +43,10 @@ static int check_removed(const struct cache_entry *ce, struct stat *st)\n>  {\n>  \tint stat_err;\n>  \n> -\tif (!(ce->ce_flags & CE_FSMONITOR_VALID))\n> -\t\tstat_err = lstat(ce->name, st);\n> -\telse\n> -\t\tstat_err = fake_lstat(ce, st);\n> +\tif (ce->ce_flags & CE_FSMONITOR_VALID)\n> +\t\treturn fake_lstat(ce, st);\n> +\n> +\tstat_err = lstat(ce->name, st);\n>  \tif (stat_err < 0) {\n>  \t\tif (!is_missing_file_error(errno))\n>  \t\t\treturn -1;\n\nNow that stat_err is used only once, we can retire the variable and\ninline the lstat() call inside the if() condition.\n\nOther than that, assuming that the CE_FSMONITOR_VALID bit is\ntrustworthy, the change looks OK.  has_symlink_leading_path() would\nbe fairly expensive and being able to avoid it is a big plus, which\nis done in other code paths like read-cache.c:refresh_cache_ent()\nand preload-index.c:preload_thread() already.\n\n\n"},{"id":"558783","messageId":"pull.2318.v2.git.1795067037014.gitgitgadget@gmail.com","threadId":"66873","inReplyTo":"pull.2318.git.1795026168007.gitgitgadget@gmail.com","subject":"[PATCH v2] diff-lib: skip leading path check for fsmonitor-valid entries","fromName":"Maik Riechert via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-11-19T05:43:56Z","receivedAt":"2026-11-19T05:43:56Z","isPatch":true,"sender":{"key":"name:Maik Riechert","avatar":null},"body":"From: Maik Riechert <letmaik@outlook.com>\n\nSince 4f3d6d0261 (fsmonitor: skip lstat deletion check during git\ndiff-index, 2021-03-17), check_removed() does not lstat() entries\nthat fsmonitor says are unchanged. But it still calls\nhas_symlink_leading_path() for them, which lstat()s the directories\nleading up to each path.\n\nThat check is there to notice a leading directory that was replaced\nby a symlink. fsmonitor would have reported such a change, and\nrefresh_fsmonitor() would have cleared CE_FSMONITOR_VALID for every\nentry below it, so we would not get here. preload_index() already\ntrusts the flag this way and skips these entries altogether.\n\nThe S_ISDIR() check that follows cannot trigger either: the mode\ncomes from the index, and gitlinks are never marked valid.\n\nSo return as soon as fake_lstat() has filled in the stat data.\n\nThe first new test checks the shortcut itself by replacing a parent\ndirectory with a symlink without telling fsmonitor. The other two make\nsure we still notice when fsmonitor does report that directory, with\nand without a trailing slash, since nothing else checks for it now.\n\nOn Linux, with 100,000 files (ten per directory), a hook that\nreports nothing and a warm cache, \"git diff HEAD\" goes from 57ms to\n26ms.\n\nSigned-off-by: Maik Riechert <letmaik@outlook.com>\n---\n    diff-lib: skip leading path check for fsmonitor-valid entries\n    \n    Changes since v1:\n    \n     * Dropped the stat_err variable and called lstat() directly in the if()\n       condition (suggested by Junio).\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-2318%2Fletmaik%2Fdiff-lib-fsmonitor-leading-path-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2318/letmaik/diff-lib-fsmonitor-leading-path-v2\nPull-Request: https://github.com/gitgitgadget/git/pull/2318\n\nRange-diff vs v1:\n\n 1:  22f99f7189 ! 1:  4f8cbd869d diff-lib: skip leading path check for fsmonitor-valid entries\n     @@ Commit message\n          Signed-off-by: Maik Riechert <letmaik@outlook.com>\n      \n       ## diff-lib.c ##\n     -@@ diff-lib.c: static int check_removed(const struct cache_entry *ce, struct stat *st)\n     +@@\n     +  */\n     + static int check_removed(const struct cache_entry *ce, struct stat *st)\n       {\n     - \tint stat_err;\n     +-\tint stat_err;\n     ++\tif (ce->ce_flags & CE_FSMONITOR_VALID)\n     ++\t\treturn fake_lstat(ce, st);\n       \n      -\tif (!(ce->ce_flags & CE_FSMONITOR_VALID))\n      -\t\tstat_err = lstat(ce->name, st);\n      -\telse\n      -\t\tstat_err = fake_lstat(ce, st);\n     -+\tif (ce->ce_flags & CE_FSMONITOR_VALID)\n     -+\t\treturn fake_lstat(ce, st);\n     -+\n     -+\tstat_err = lstat(ce->name, st);\n     - \tif (stat_err < 0) {\n     +-\tif (stat_err < 0) {\n     ++\tif (lstat(ce->name, st) < 0) {\n       \t\tif (!is_missing_file_error(errno))\n       \t\t\treturn -1;\n     + \t\treturn 1;\n      \n       ## t/t7519-status-fsmonitor.sh ##\n      @@ t/t7519-status-fsmonitor.sh: do\n\n\n diff-lib.c                  |  9 +++-----\n t/t7519-status-fsmonitor.sh | 44 +++++++++++++++++++++++++++++++++++++\n 2 files changed, 47 insertions(+), 6 deletions(-)\n\ndiff --git a/diff-lib.c b/diff-lib.c\nindex 086476bd77..b644154397 100644\n--- a/diff-lib.c\n+++ b/diff-lib.c\n@@ -41,13 +41,10 @@\n  */\n static int check_removed(const struct cache_entry *ce, struct stat *st)\n {\n-\tint stat_err;\n+\tif (ce->ce_flags & CE_FSMONITOR_VALID)\n+\t\treturn fake_lstat(ce, st);\n \n-\tif (!(ce->ce_flags & CE_FSMONITOR_VALID))\n-\t\tstat_err = lstat(ce->name, st);\n-\telse\n-\t\tstat_err = fake_lstat(ce, st);\n-\tif (stat_err < 0) {\n+\tif (lstat(ce->name, st) < 0) {\n \t\tif (!is_missing_file_error(errno))\n \t\t\treturn -1;\n \t\treturn 1;\ndiff --git a/t/t7519-status-fsmonitor.sh b/t/t7519-status-fsmonitor.sh\nindex 93973ed25a..6c9df9f333 100755\n--- a/t/t7519-status-fsmonitor.sh\n+++ b/t/t7519-status-fsmonitor.sh\n@@ -336,6 +336,50 @@ do\n \tdone\n done\n \n+# Create a repository whose only entry, dir1/tracked, is marked valid\n+# by an fsmonitor hook that reports no changes, then replace dir1 with\n+# a symlink to a directory that has the same contents.\n+setup_symlinked_leading_dir () {\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tmkdir repo/dir1 &&\n+\t: >repo/dir1/tracked &&\n+\tgit -C repo add dir1/tracked &&\n+\tgit -C repo commit -m initial &&\n+\ttest_hook -C repo --setup fsmonitor-test <<-\\EOF &&\n+\tprintf \"last_update_token\\0\"\n+\tEOF\n+\tgit -C repo config core.fsmonitor .git/hooks/fsmonitor-test &&\n+\tgit -C repo update-index --refresh --force-write-index &&\n+\techo \"h dir1/tracked\" >expect &&\n+\tgit -C repo ls-files -f >actual &&\n+\ttest_cmp expect actual &&\n+\tmv repo/dir1 repo/dir2 &&\n+\tln -s dir2 repo/dir1\n+}\n+\n+# Like \"status\" above, diff-index should trust fsmonitor and not look\n+# for symlinks in leading directories that it was not told about.\n+test_expect_success SYMLINKS \"diff-index doesn't detect unreported symlinked directories\" '\n+\tsetup_symlinked_leading_dir &&\n+\tgit -C repo diff-index --name-status HEAD >actual &&\n+\ttest_must_be_empty actual\n+'\n+\n+for report in dir1 dir1/\n+do\n+\ttest_expect_success SYMLINKS \"diff-index detects symlinked directory reported as $report\" '\n+\t\tsetup_symlinked_leading_dir &&\n+\t\ttest_hook -C repo --clobber fsmonitor-test <<-EOF &&\n+\t\tprintf \"last_update_token\\0\"\n+\t\tprintf \"$report\\0\"\n+\t\tEOF\n+\t\tprintf \"D\\tdir1/tracked\\n\" >expect &&\n+\t\tgit -C repo diff-index --name-status HEAD >actual &&\n+\t\ttest_cmp expect actual\n+\t'\n+done\n+\n # test that splitting the index doesn't interfere\n test_expect_success 'splitting the index results in the same state' '\n \twrite_integration_script &&\n\nbase-commit: e1ec09b23c8e0deb21a0b95ac5996422277395d4\n-- \ngitgitgadget\n\n"},{"id":"558893","messageId":"xmqqv75sv5li.fsf@gitster.g","threadId":"66873","inReplyTo":"pull.2318.v2.git.1795067037014.gitgitgadget@gmail.com","subject":"Re: [PATCH v2] diff-lib: skip leading path check for fsmonitor-valid entries","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-11-19T22:06:33Z","receivedAt":"2026-11-19T22:06:33Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Maik Riechert via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> From: Maik Riechert <letmaik@outlook.com>\n>\n> Since 4f3d6d0261 (fsmonitor: skip lstat deletion check during git\n> diff-index, 2021-03-17), check_removed() does not lstat() entries\n> that fsmonitor says are unchanged. But it still calls\n> has_symlink_leading_path() for them, which lstat()s the directories\n> leading up to each path.\n>\n> That check is there to notice a leading directory that was replaced\n> by a symlink. fsmonitor would have reported such a change, and\n> refresh_fsmonitor() would have cleared CE_FSMONITOR_VALID for every\n> entry below it, so we would not get here. preload_index() already\n> trusts the flag this way and skips these entries altogether.\n>\n> The S_ISDIR() check that follows cannot trigger either: the mode\n> comes from the index, and gitlinks are never marked valid.\n\nThis is slightly misleading.  S_IFGITLINK is given S_IFDIR | 0755\nby st_mode_from_ce() that is called by fake_lstat(), so S_ISDIR()\ncheck *does* trigger.  We have an extra S_ISGITLINK() guard in\nthere, preventing us from calling repo_resolve_gitlink_ref().\n\nIt does not change the conclusion, though.\n\n> So return as soon as fake_lstat() has filled in the stat data.\n>\n> The first new test checks the shortcut itself by replacing a parent\n> directory with a symlink without telling fsmonitor. The other two make\n> sure we still notice when fsmonitor does report that directory, with\n> and without a trailing slash, since nothing else checks for it now.\n>\n> On Linux, with 100,000 files (ten per directory), a hook that\n> reports nothing and a warm cache, \"git diff HEAD\" goes from 57ms to\n> 26ms.\n>\n> Signed-off-by: Maik Riechert <letmaik@outlook.com>\n> ---\n\nWill queue.  Thanks.\n\n"}]}