Re: [PATCH] ls-files: filter pathspec before lstat
- From
Tamir Duberstein <tamird@gmail.com>
- Date
- Jun 8, 2026, 19:15 UTC
- Message-ID
- <CAJ-ks9njUM4TqHd=3H+aY8TCk6yG4o1yAhiSn2Tfz6oDnML20A@mail.gmail.com>
- In-Reply-To
- <xmqqa4t5yyee.fsf@gitster.g>
On Mon, Jun 8, 2026 at 6:06 AM Junio C Hamano <gitster@pobox.com> wrote:
Show 18 quoted lines
> > On Sun, Jun 7, 2026 at 11:40, Tamir Duberstein wrote: > > show_files() checks whether each index entry is deleted or modified > > before show_ce() applies the pathspec. prune_index() avoids most of this > > work for pathspecs with a common directory prefix, but a top-level name > > or leading wildcard leaves every entry to be checked. > > > > Match the pathspec before lstat() for the deleted and modified modes. > > Keep the later match in show_ce() so --error-unmatch is satisfied only > > by entries that are actually shown. > > Adding an extra early `match_pathspec()` check before making slow > system calls like `lstat()` makes sense, especially when most of the > index entries need to be skipped. But if most of them would match, > then we would end up doing the same match_pathspec() calls twice for > each path, and run lstat() anyway, so you may also be able to > construct a perf test that demonstrates a case where this approach > is not a clear win (or even degradation), perhaps?
Yes. I added an all-matching pathspec case to p3010 and ran:
hyperfine --warmup 0 --runs 3 \
'git -c core.fsmonitor=false ls-files --deleted -- "*"'On a checkout with 859,940 index entries, I ran the parent and patched binaries in both orders:
parent this commit
parent first elapsed 56.807 s 64.618 s
user 1.256 s 1.270 s
system 10.633 s 11.068 s
patched first elapsed 63.361 s 64.316 s
user 1.238 s 1.280 s
system 10.296 s 11.864 sThe added match costs 14-42 ms of user time in this case. Elapsed time varies by several seconds with command order, obscuring that CPU cost.
The later match in show_ce() is reached only for entries actually found deleted or modified. This case therefore exercises the extra match for every index entry while still performing every lstat().
Show 26 quoted lines
>
> > diff --git a/builtin/ls-files.c b/builtin/ls-files.c
> > index e1a22b41b9..702c607183 100644
> > --- a/builtin/ls-files.c
> > +++ b/builtin/ls-files.c
> > @@ -450,6 +450,13 @@ static void show_files(struct repository *repo, struct dir_struct *dir)
> > continue;
> > if (ce_skip_worktree(ce))
> > continue;
> > + /* Only entries shown by show_ce() satisfy --error-unmatch. */
> > + if (pathspec.nr &&
> > + !match_pathspec(repo->index, &pathspec, fullname.buf,
> > + fullname.len, max_prefix_len, NULL,
> > + S_ISDIR(ce->ce_mode) ||
> > + S_ISGITLINK(ce->ce_mode)))
> > + continue;
> > stat_err = lstat(fullname.buf, &st);
> > if (stat_err && (errno != ENOENT && errno != ENOTDIR))
> > error_errno("cannot lstat '%s'", fullname.buf);
>
> Hmph. In the current code, because there is no such pre-filtering,
> show_ce() would unconditionally recurse into active submodules when
> told to with the "--recurse-submodules" flag, even if your pathspec
> coes not match the submodule. With this change, such a submodule
> whose path does not match the pathspec would not even be seen by
> show_ce(). Would it cause a change in behaviour?This path cannot affect --recurse-submodules. cmd_ls_files() rejects --recurse-submodules together with either --deleted or --modified before calling show_files(), and the new check is reached only for those two modes. Cached and stage output continue to call show_ce() before the new check. t3007 already verifies that both combinations are rejected.
Given the 60.742 s to 1.061 s improvement for a selective pathspec, I think this small CPU cost for an all-matching pathspec is a worthwhile tradeoff. What do you think?
Thanks for the review! Tamir