threads / discuss / 66791

git-ls-files(1) avoiding symlinks

Subject: git-ls-files(1) avoiding symlinks

## tl;dr

6 messages between Nov 6, 2026 and Nov 6, 2026.

replies: 5people: 3as markdown or json

Alejandro Colomar· Nov 6, 2026, 13:35 UTC · lore
Hi!

Yesterday, we were discussing alternative methods for grepping source code. Here's a list of similar alternatives:

	$ find . -type f | xargs grep ARRAY_END;
	$ git ls-files | xargs grep ARRAY_END;
	$ git grep ARRAY_END;

The version using find(1) has some problems: it is slow, especially after having built a kernel, when the directory is full of temporary objects.

The version using git-grep(1) without IPC is the fastest, and doesn't have any problems, but it is limited to just grep(1). If we want to use sed(1), or any other programs, we're out of luck.

A common program I want to use is grepc(1), which searches source code. See for example this code for finding the creat(2) system call:

	$ git ls-files | grep '\.[ch]$' | xargs grepc creat;
	fs/open.c:SYSCALL_DEFINE2(creat, const char __user *, pathname, umode_t, mode)
	{
		int flags = O_CREAT | O_WRONLY | O_TRUNC;
		if (force_o_largefile())
			flags |= O_LARGEFILE;
		return do_sys_open(AT_FDCWD, pathname, flags, mode);
	}
	include/linux/syscalls.h:asmlinkage long sys_creat(const char __user *pathname, umode_t mode);
	tools/include/nolibc/fcntl.h:static __attribute__((unused))
	int creat(const char *path, mode_t mode)
	{
		return open(path, O_CREAT | O_WRONLY | O_TRUNC, mode);
	}

A problem with git-ls-files(1) is that it doesn't have a way to filter files by type. That is, it will list symlinks, which will affect the search. With find(1), I can control that with '-type f', which makes sure I only search in regular files. I'd like to have some feature in git-ls-files(1) that allows me to print files of the specified type. Since git(1) knows the type of the files, this should be doable without much overhead, right?

Have a lovely day! Alex

-- 
<https://www.alejandro-colomar.es>
D. Ben Knoble· Nov 6, 2026, 14:15 UTC · re: Alejandro Colomar · lore

Re: git-ls-files(1) avoiding symlinks

On Fri, Nov 6, 2026 at 8:35 AM Alejandro Colomar <alx+git@kernel.org> wrote:
Show 44 quoted lines
>
> Hi!
>
> Yesterday, we were discussing alternative methods for grepping source
> code.  Here's a list of similar alternatives:
>
>         $ find . -type f | xargs grep ARRAY_END;
>         $ git ls-files | xargs grep ARRAY_END;
>         $ git grep ARRAY_END;
>
> The version using find(1) has some problems: it is slow, especially
> after having built a kernel, when the directory is full of temporary
> objects.
>
> The version using git-grep(1) without IPC is the fastest, and doesn't
> have any problems, but it is limited to just grep(1).  If we want to use
> sed(1), or any other programs, we're out of luck.
>
> A common program I want to use is grepc(1), which searches source code.
> See for example this code for finding the creat(2) system call:
>
>         $ git ls-files | grep '\.[ch]$' | xargs grepc creat;
>         fs/open.c:SYSCALL_DEFINE2(creat, const char __user *, pathname, umode_t, mode)
>         {
>                 int flags = O_CREAT | O_WRONLY | O_TRUNC;
>
>                 if (force_o_largefile())
>                         flags |= O_LARGEFILE;
>                 return do_sys_open(AT_FDCWD, pathname, flags, mode);
>         }
>         include/linux/syscalls.h:asmlinkage long sys_creat(const char __user *pathname, umode_t mode);
>         tools/include/nolibc/fcntl.h:static __attribute__((unused))
>         int creat(const char *path, mode_t mode)
>         {
>                 return open(path, O_CREAT | O_WRONLY | O_TRUNC, mode);
>         }
>
> A problem with git-ls-files(1) is that it doesn't have a way to filter
> files by type.  That is, it will list symlinks, which will affect the
> search.  With find(1), I can control that with '-type f', which makes
> sure I only search in regular files.  I'd like to have some feature in
> git-ls-files(1) that allows me to print files of the specified type.
> Since git(1) knows the type of the files, this should be doable without
> much overhead, right?

Perhaps something with "git ls-files --format='%(objectmode) %(path)' -z"? Of course, your examples already omit -z, so if you can assume that it might be simpler.

-- 
D. Ben Knoble
Alejandro Colomar· Nov 6, 2026, 14:45 UTC · re: D. Ben Knoble · lore

Re: git-ls-files(1) avoiding symlinks

Hi Ben,
> Date: 2026-11-06 09:15:22-0500
> From: "D. Ben Knoble" <ben.knoble@gmail.com>
>
[...]
Show 11 quoted lines
> > A problem with git-ls-files(1) is that it doesn't have a way to filter
> > files by type.  That is, it will list symlinks, which will affect the
> > search.  With find(1), I can control that with '-type f', which makes
> > sure I only search in regular files.  I'd like to have some feature in
> > git-ls-files(1) that allows me to print files of the specified type.
> > Since git(1) knows the type of the files, this should be doable without
> > much overhead, right?
> 
> Perhaps something with "git ls-files --format='%(objectmode) %(path)'
> -z"? Of course, your examples already omit -z, so if you can assume
> that it might be simpler.
Hmmm, that could work.  Thanks!
	$ git ls-files --format='%(objectmode) %(path)' \
	| grep -v ^120000 \
	| sed 's/^.* //' \
	| grep '\.[ch]$' \
	| xargs grepc creat;
With -z it becomes a bit tedious:
	$ git ls-files -z --format='%(objectmode) %(path)' \
	| grep -zv ^120000 \
	| sed -z 's/^.* //' \
	| grep -z '\.[ch]$' \
	| xargs -0 grepc creat;

But, indeed, I don't care about files with spaces often, because I don't create them in my stuff.

I expect git-ls-files(1) could make this much simpler, though:
	$ git ls-files --type f \
	| grep '\.[ch]$' \
	| xargs grepc creat;

Have a lovely day! Alex

-- 
<https://www.alejandro-colomar.es>
Alejandro Colomar· Nov 6, 2026, 19:36 UTC · lore

Re: git-ls-files(1) avoiding symlinks

Hi Ben,
> Date: 2026-11-06 12:34:20-0500
> From: Ben Knoble <ben.knoble@gmail.com>
>
[...]
Show 19 quoted lines
> > Hmmm, that could work.  Thanks!
> > 
> >    $ git ls-files --format='%(objectmode) %(path)' \
> >    | grep -v ^120000 \
> >    | sed 's/^.* //' \
> >    | grep '\.[ch]$' \
> >    | xargs grepc creat;
> > 
> > With -z it becomes a bit tedious:
> > 
> >    $ git ls-files -z --format='%(objectmode) %(path)' \
> >    | grep -zv ^120000 \
> >    | sed -z 's/^.* //' \
> >    | grep -z '\.[ch]$' \
> >    | xargs -0 grepc creat;
> 
> The filtering for C files can also be done by Git, I believe:
> 
>     git ls-files … '*.[ch]'

I prefer regexes over globs; they're more powerful, and they're hot in brain memory. :)

Show 10 quoted lines
> > But, indeed, I don't care about files with spaces often, because I don't
> > create them in my stuff.
> > 
> > I expect git-ls-files(1) could make this much simpler, though:
> > 
> >    $ git ls-files --type f \
> >    | grep '\.[ch]$' \
> >    | xargs grepc creat;
> 
> I believe the expression is “patches [scratching your own itch are] welcome” ;) 
Nice!  If the idea is welcome, I can certainly work on a patch.  :)

Have a lovely night! Alex

-- 
<https://www.alejandro-colomar.es>
Junio C Hamano· Nov 6, 2026, 20:16 UTC · re: Alejandro Colomar · lore

Re: git-ls-files(1) avoiding symlinks

Alejandro Colomar <alx+git@kernel.org> writes:
Show 7 quoted lines
>> > With -z it becomes a bit tedious:
>> > 
>> >    $ git ls-files -z --format='%(objectmode) %(path)' \
>> >    | grep -zv ^120000 \
>> >    | sed -z 's/^.* //' \
>> >    | grep -z '\.[ch]$' \
>> >    | xargs -0 grepc creat;
Shouldn't the grep|sed|grep be doable with a single sed invocation?
Show 5 quoted lines
>> > I expect git-ls-files(1) could make this much simpler, though:
>> > 
>> >    $ git ls-files --type f \
>> >    | grep '\.[ch]$' \
>> >    | xargs grepc creat;

Not really. Why is "--type" have to be so special? What would the next person ask for? --size?

The "ls-files" plumbing command is designed as a component to build your own pipeline with. Don't throw in random unrelated features to it. In the longer run, it is not a good substitute for improving the skill to construct a good pipeline.

Learn the tools available to you that you already have heard of, so that you can make better uses of them (in this case "sed"). If you squash the grep|sed|grep pipeline into a single sed

    git ls-files -z -s "*.[ch]" |
    sed -z -n -e '/^120000/d' -e '/^.* 0	/s///p' |
    xargs -r -0 ls -ld
the result does not look too bad.

One unrelated tip for better readability is to end, not begin, your lines with the "|" pipe characters. That way you do not have to sprinkle so many unsightly backslashes ;-).

HTH.
Alejandro Colomar· Nov 6, 2026, 21:06 UTC · re: Junio C Hamano · lore

Re: git-ls-files(1) avoiding symlinks

Hi Junio,
Show 23 quoted lines
> Date: 2026-11-06 12:16:48-0800
> From: Junio C Hamano <gitster@pobox.com>
>
> Alejandro Colomar <alx+git@kernel.org> writes:
> 
> >> > With -z it becomes a bit tedious:
> >> > 
> >> >    $ git ls-files -z --format='%(objectmode) %(path)' \
> >> >    | grep -zv ^120000 \
> >> >    | sed -z 's/^.* //' \
> >> >    | grep -z '\.[ch]$' \
> >> >    | xargs -0 grepc creat;
> 
> Shouldn't the grep|sed|grep be doable with a single sed invocation?
> 
> >> > I expect git-ls-files(1) could make this much simpler, though:
> >> > 
> >> >    $ git ls-files --type f \
> >> >    | grep '\.[ch]$' \
> >> >    | xargs grepc creat;
> 
> Not really.  Why is "--type" have to be so special?  What would the
> next person ask for?  --size?

Indeed. While looking at the implementation of git-ls-files(1), I realized it wasn't easy to decide how to implement this, as it has other types (objecttype, which is blob/commit/...). I thought perhaps --objectmode, but then I'm not inderested in searching specific modes, but rather filtering out some. Perhaps in this case it's more appropriate to have my own local command 'git-find' that does what I want.

I've ended up doing this, and now don't think a flag in git-ls-files(1) would make much sense.

	$ cat $(which git-find)
	#!/bin/bash
	# Copyright 2026, Alejandro Colomar <alx@kernel.org>
	# SPDX-License-Identifier: GPL-3.0-or-later
	set -Eeufo pipefail;
	git ls-files --format='%(objectmode) %(path)' \
	| grep -v '^120000' \
	| sed 's/^.* //';

And will eventually add a -z flag for this command when I find I need it.

> The "ls-files" plumbing command is designed as a component to build
> your own pipeline with.  Don't throw in random unrelated features to
> it.  In the longer run, it is not a good substitute for improving
> the skill to construct a good pipeline.
Indeed.
Show 9 quoted lines
> Learn the tools available to you that you already have heard of, so
> that you can make better uses of them (in this case "sed").  If you
> squash the grep|sed|grep pipeline into a single sed
> 
>     git ls-files -z -s "*.[ch]" |
>     sed -z -n -e '/^120000/d' -e '/^.* 0	/s///p' |
>     xargs -r -0 ls -ld
> 
> the result does not look too bad.

I find it simpler to have a simple grep -v rejecting the lines I don't want, and then sed(1) getting rid of the stuff. The command above seems interesting for performance, but compared to the tool using the file names, that's probably negligible.

About git-ls-files(1)'s -s, I think it's easier to parse with the %(objectmode) in the format, as I don't need all the other information printed by -s.

> 
> One unrelated tip for better readability is to end, not begin, your
> lines with the "|" pipe characters.  That way you do not have to
> sprinkle so many unsightly backslashes ;-).

I like to have them at the front of the line because it shows more clearly whether I'm piping a command or starting a new pipeline. I got used to the backslashes. After all, the first few characters of a line are all aligned, and the most significant ones. Stuff to the right I tend to ignore it as noise. :)

> 
> HTH.
Yup!  Thanks!

Have a lovely night! Alex

-- 
<https://www.alejandro-colomar.es>

← back to recent threads