git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] ls-files: filter pathspec before lstat

From
Tamir Duberstein <tamird@gmail.com>
Date
Jun 8, 2026, 19:15 UTC
Message-ID
<CAJ-ks9njUM4TqHd=3H+aY8TCk6yG4o1yAhiSn2Tfz6oDnML20A@mail.gmail.com>
In-Reply-To
<xmqqa4t5yyee.fsf@gitster.g>
On Mon, Jun 8, 2026 at 6:06 AM Junio C Hamano <gitster@pobox.com> wrote:
Show 18 quoted lines
>
> On Sun, Jun 7, 2026 at 11:40, Tamir Duberstein wrote:
> > show_files() checks whether each index entry is deleted or modified
> > before show_ce() applies the pathspec. prune_index() avoids most of this
> > work for pathspecs with a common directory prefix, but a top-level name
> > or leading wildcard leaves every entry to be checked.
> >
> > Match the pathspec before lstat() for the deleted and modified modes.
> > Keep the later match in show_ce() so --error-unmatch is satisfied only
> > by entries that are actually shown.
>
> Adding an extra early `match_pathspec()` check before making slow
> system calls like `lstat()` makes sense, especially when most of the
> index entries need to be skipped.  But if most of them would match,
> then we would end up doing the same match_pathspec() calls twice for
> each path, and run lstat() anyway, so you may also be able to
> construct a perf test that demonstrates a case where this approach
> is not a clear win (or even degradation), perhaps?
Yes. I added an all-matching pathspec case to p3010 and ran:
    hyperfine --warmup 0 --runs 3 \
        'git -c core.fsmonitor=false ls-files --deleted -- "*"'

On a checkout with 859,940 index entries, I ran the parent and patched binaries in both orders:

                         parent          this commit
  parent first elapsed    56.807 s        64.618 s
               user        1.256 s         1.270 s
               system     10.633 s        11.068 s
  patched first elapsed   63.361 s        64.316 s
                user       1.238 s         1.280 s
                system    10.296 s        11.864 s

The added match costs 14-42 ms of user time in this case. Elapsed time varies by several seconds with command order, obscuring that CPU cost.

The later match in show_ce() is reached only for entries actually found deleted or modified. This case therefore exercises the extra match for every index entry while still performing every lstat().

Show 26 quoted lines
>
> > diff --git a/builtin/ls-files.c b/builtin/ls-files.c
> > index e1a22b41b9..702c607183 100644
> > --- a/builtin/ls-files.c
> > +++ b/builtin/ls-files.c
> > @@ -450,6 +450,13 @@ static void show_files(struct repository *repo, struct dir_struct *dir)
> >                       continue;
> >               if (ce_skip_worktree(ce))
> >                       continue;
> > +             /* Only entries shown by show_ce() satisfy --error-unmatch. */
> > +             if (pathspec.nr &&
> > +                 !match_pathspec(repo->index, &pathspec, fullname.buf,
> > +                                 fullname.len, max_prefix_len, NULL,
> > +                                 S_ISDIR(ce->ce_mode) ||
> > +                                 S_ISGITLINK(ce->ce_mode)))
> > +                     continue;
> >               stat_err = lstat(fullname.buf, &st);
> >               if (stat_err && (errno != ENOENT && errno != ENOTDIR))
> >                       error_errno("cannot lstat '%s'", fullname.buf);
>
> Hmph.  In the current code, because there is no such pre-filtering,
> show_ce() would unconditionally recurse into active submodules when
> told to with the "--recurse-submodules" flag, even if your pathspec
> coes not match the submodule.  With this change, such a submodule
> whose path does not match the pathspec would not even be seen by
> show_ce().  Would it cause a change in behaviour?

This path cannot affect --recurse-submodules. cmd_ls_files() rejects --recurse-submodules together with either --deleted or --modified before calling show_files(), and the new check is reached only for those two modes. Cached and stage output continue to call show_ce() before the new check. t3007 already verifies that both combinations are rejected.

Given the 60.742 s to 1.061 s improvement for a selective pathspec, I think this small CPU cost for an all-matching pathspec is a worthwhile tradeoff. What do you think?

Thanks for the review! Tamir

Previous: Junio C HamanoNext: Jeff King
Message 8 of 11 in “ls-files: filter pathspec before lstat”
  1. ls-files: filter pathspec before lstatTamir Duberstein, Jun 7, 2026
  2. Kristoffer HaugsbakkJun 7, 2026
  3. Tamir DubersteinJun 7, 2026
  4. Kristoffer HaugsbakkJun 7, 2026
  5. Tamir DubersteinJun 7, 2026
  6. Kristoffer HaugsbakkSep 6, 2026
  7. Junio C HamanoJun 8, 2026
  8. Tamir DubersteinJun 8, 2026
  9. Jeff KingJun 8, 2026
  10. Jeff KingJun 8, 2026
  11. Tamir DubersteinJun 9, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.