{"thread":{"id":"66791","subject":"git-ls-files(1) avoiding symlinks","startedAt":"2026-11-06T13:35:01Z","lastAt":"2026-11-06T21:06:38Z","messageCount":6,"participants":["Alejandro Colomar","D. Ben Knoble","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"557677","messageId":"au3Tyaz_LoYDuskn@devuan","threadId":"66791","inReplyTo":null,"subject":"git-ls-files(1) avoiding symlinks","fromName":"Alejandro Colomar","fromEmail":"alx+git@kernel.org","sentAt":"2026-11-06T13:35:01Z","receivedAt":"2026-11-06T13:35:01Z","isPatch":false,"sender":{"key":"alx+git@kernel.org","avatar":null},"body":"Hi!\n\nYesterday, we were discussing alternative methods for grepping source\ncode.  Here's a list of similar alternatives:\n\n\t$ find . -type f | xargs grep ARRAY_END;\n\t$ git ls-files | xargs grep ARRAY_END;\n\t$ git grep ARRAY_END;\n\nThe version using find(1) has some problems: it is slow, especially\nafter having built a kernel, when the directory is full of temporary\nobjects.\n\nThe version using git-grep(1) without IPC is the fastest, and doesn't\nhave any problems, but it is limited to just grep(1).  If we want to use\nsed(1), or any other programs, we're out of luck.\n\nA common program I want to use is grepc(1), which searches source code.\nSee for example this code for finding the creat(2) system call:\n\n\t$ git ls-files | grep '\\.[ch]$' | xargs grepc creat;\n\tfs/open.c:SYSCALL_DEFINE2(creat, const char __user *, pathname, umode_t, mode)\n\t{\n\t\tint flags = O_CREAT | O_WRONLY | O_TRUNC;\n\n\t\tif (force_o_largefile())\n\t\t\tflags |= O_LARGEFILE;\n\t\treturn do_sys_open(AT_FDCWD, pathname, flags, mode);\n\t}\n\tinclude/linux/syscalls.h:asmlinkage long sys_creat(const char __user *pathname, umode_t mode);\n\ttools/include/nolibc/fcntl.h:static __attribute__((unused))\n\tint creat(const char *path, mode_t mode)\n\t{\n\t\treturn open(path, O_CREAT | O_WRONLY | O_TRUNC, mode);\n\t}\n\nA problem with git-ls-files(1) is that it doesn't have a way to filter\nfiles by type.  That is, it will list symlinks, which will affect the\nsearch.  With find(1), I can control that with '-type f', which makes\nsure I only search in regular files.  I'd like to have some feature in\ngit-ls-files(1) that allows me to print files of the specified type.\nSince git(1) knows the type of the files, this should be doable without\nmuch overhead, right?\n\n\nHave a lovely day!\nAlex\n\n-- \n<https://www.alejandro-colomar.es>\n"},{"id":"557684","messageId":"CALnO6CB-b2XuvwkQ9pJp5ENuc12oGMdgbcwz1VKD_i3KWfKuHA@mail.gmail.com","threadId":"66791","inReplyTo":"au3Tyaz_LoYDuskn@devuan","subject":"Re: git-ls-files(1) avoiding symlinks","fromName":"D. Ben Knoble","fromEmail":"ben.knoble@gmail.com","sentAt":"2026-11-06T14:15:22Z","receivedAt":"2026-11-06T14:15:22Z","isPatch":false,"sender":{"key":"ben.knoble@gmail.com","avatar":"https://avatars.githubusercontent.com/u/22802209?v=4"},"body":"On Fri, Nov 6, 2026 at 8:35 AM Alejandro Colomar <alx+git@kernel.org> wrote:\n>\n> Hi!\n>\n> Yesterday, we were discussing alternative methods for grepping source\n> code.  Here's a list of similar alternatives:\n>\n>         $ find . -type f | xargs grep ARRAY_END;\n>         $ git ls-files | xargs grep ARRAY_END;\n>         $ git grep ARRAY_END;\n>\n> The version using find(1) has some problems: it is slow, especially\n> after having built a kernel, when the directory is full of temporary\n> objects.\n>\n> The version using git-grep(1) without IPC is the fastest, and doesn't\n> have any problems, but it is limited to just grep(1).  If we want to use\n> sed(1), or any other programs, we're out of luck.\n>\n> A common program I want to use is grepc(1), which searches source code.\n> See for example this code for finding the creat(2) system call:\n>\n>         $ git ls-files | grep '\\.[ch]$' | xargs grepc creat;\n>         fs/open.c:SYSCALL_DEFINE2(creat, const char __user *, pathname, umode_t, mode)\n>         {\n>                 int flags = O_CREAT | O_WRONLY | O_TRUNC;\n>\n>                 if (force_o_largefile())\n>                         flags |= O_LARGEFILE;\n>                 return do_sys_open(AT_FDCWD, pathname, flags, mode);\n>         }\n>         include/linux/syscalls.h:asmlinkage long sys_creat(const char __user *pathname, umode_t mode);\n>         tools/include/nolibc/fcntl.h:static __attribute__((unused))\n>         int creat(const char *path, mode_t mode)\n>         {\n>                 return open(path, O_CREAT | O_WRONLY | O_TRUNC, mode);\n>         }\n>\n> A problem with git-ls-files(1) is that it doesn't have a way to filter\n> files by type.  That is, it will list symlinks, which will affect the\n> search.  With find(1), I can control that with '-type f', which makes\n> sure I only search in regular files.  I'd like to have some feature in\n> git-ls-files(1) that allows me to print files of the specified type.\n> Since git(1) knows the type of the files, this should be doable without\n> much overhead, right?\n\nPerhaps something with \"git ls-files --format='%(objectmode) %(path)'\n-z\"? Of course, your examples already omit -z, so if you can assume\nthat it might be simpler.\n\n-- \nD. Ben Knoble\n\n"},{"id":"557689","messageId":"au3lu4jYwZ5hBZSy@devuan","threadId":"66791","inReplyTo":"CALnO6CB-b2XuvwkQ9pJp5ENuc12oGMdgbcwz1VKD_i3KWfKuHA@mail.gmail.com","subject":"Re: git-ls-files(1) avoiding symlinks","fromName":"Alejandro Colomar","fromEmail":"alx+git@kernel.org","sentAt":"2026-11-06T14:45:08Z","receivedAt":"2026-11-06T14:45:08Z","isPatch":false,"sender":{"key":"alx+git@kernel.org","avatar":null},"body":"Hi Ben,\n\n> Date: 2026-11-06 09:15:22-0500\n> From: \"D. Ben Knoble\" <ben.knoble@gmail.com>\n>\n[...]\n> > A problem with git-ls-files(1) is that it doesn't have a way to filter\n> > files by type.  That is, it will list symlinks, which will affect the\n> > search.  With find(1), I can control that with '-type f', which makes\n> > sure I only search in regular files.  I'd like to have some feature in\n> > git-ls-files(1) that allows me to print files of the specified type.\n> > Since git(1) knows the type of the files, this should be doable without\n> > much overhead, right?\n> \n> Perhaps something with \"git ls-files --format='%(objectmode) %(path)'\n> -z\"? Of course, your examples already omit -z, so if you can assume\n> that it might be simpler.\n\nHmmm, that could work.  Thanks!\n\n\t$ git ls-files --format='%(objectmode) %(path)' \\\n\t| grep -v ^120000 \\\n\t| sed 's/^.* //' \\\n\t| grep '\\.[ch]$' \\\n\t| xargs grepc creat;\n\nWith -z it becomes a bit tedious:\n\n\t$ git ls-files -z --format='%(objectmode) %(path)' \\\n\t| grep -zv ^120000 \\\n\t| sed -z 's/^.* //' \\\n\t| grep -z '\\.[ch]$' \\\n\t| xargs -0 grepc creat;\n\nBut, indeed, I don't care about files with spaces often, because I don't\ncreate them in my stuff.\n\nI expect git-ls-files(1) could make this much simpler, though:\n\n\t$ git ls-files --type f \\\n\t| grep '\\.[ch]$' \\\n\t| xargs grepc creat;\n\n\nHave a lovely day!\nAlex\n\n-- \n<https://www.alejandro-colomar.es>\n"},{"id":"557730","messageId":"au4rmQRjRtIv1Oru@debian","threadId":"66791","inReplyTo":"AE63992D-7BEA-413D-A372-78708CFA8F6F@gmail.com","subject":"Re: git-ls-files(1) avoiding symlinks","fromName":"Alejandro Colomar","fromEmail":"alx+git@kernel.org","sentAt":"2026-11-06T19:36:31Z","receivedAt":"2026-11-06T19:36:31Z","isPatch":false,"sender":{"key":"alx+git@kernel.org","avatar":null},"body":"Hi Ben,\n\n> Date: 2026-11-06 12:34:20-0500\n> From: Ben Knoble <ben.knoble@gmail.com>\n>\n[...]\n> > Hmmm, that could work.  Thanks!\n> > \n> >    $ git ls-files --format='%(objectmode) %(path)' \\\n> >    | grep -v ^120000 \\\n> >    | sed 's/^.* //' \\\n> >    | grep '\\.[ch]$' \\\n> >    | xargs grepc creat;\n> > \n> > With -z it becomes a bit tedious:\n> > \n> >    $ git ls-files -z --format='%(objectmode) %(path)' \\\n> >    | grep -zv ^120000 \\\n> >    | sed -z 's/^.* //' \\\n> >    | grep -z '\\.[ch]$' \\\n> >    | xargs -0 grepc creat;\n> \n> The filtering for C files can also be done by Git, I believe:\n> \n>     git ls-files … '*.[ch]'\n\nI prefer regexes over globs; they're more powerful, and they're hot in\nbrain memory.  :)\n\n> > But, indeed, I don't care about files with spaces often, because I don't\n> > create them in my stuff.\n> > \n> > I expect git-ls-files(1) could make this much simpler, though:\n> > \n> >    $ git ls-files --type f \\\n> >    | grep '\\.[ch]$' \\\n> >    | xargs grepc creat;\n> \n> I believe the expression is “patches [scratching your own itch are] welcome” ;) \n\nNice!  If the idea is welcome, I can certainly work on a patch.  :)\n\n\nHave a lovely night!\nAlex\n\n-- \n<https://www.alejandro-colomar.es>\n"},{"id":"557732","messageId":"xmqqeccxhg0f.fsf@gitster.g","threadId":"66791","inReplyTo":"au4rmQRjRtIv1Oru@debian","subject":"Re: git-ls-files(1) avoiding symlinks","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-11-06T20:16:48Z","receivedAt":"2026-11-06T20:16:48Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Alejandro Colomar <alx+git@kernel.org> writes:\n\n>> > With -z it becomes a bit tedious:\n>> > \n>> >    $ git ls-files -z --format='%(objectmode) %(path)' \\\n>> >    | grep -zv ^120000 \\\n>> >    | sed -z 's/^.* //' \\\n>> >    | grep -z '\\.[ch]$' \\\n>> >    | xargs -0 grepc creat;\n\nShouldn't the grep|sed|grep be doable with a single sed invocation?\n\n>> > I expect git-ls-files(1) could make this much simpler, though:\n>> > \n>> >    $ git ls-files --type f \\\n>> >    | grep '\\.[ch]$' \\\n>> >    | xargs grepc creat;\n\nNot really.  Why is \"--type\" have to be so special?  What would the\nnext person ask for?  --size?\n\nThe \"ls-files\" plumbing command is designed as a component to build\nyour own pipeline with.  Don't throw in random unrelated features to\nit.  In the longer run, it is not a good substitute for improving\nthe skill to construct a good pipeline.\n\nLearn the tools available to you that you already have heard of, so\nthat you can make better uses of them (in this case \"sed\").  If you\nsquash the grep|sed|grep pipeline into a single sed\n\n    git ls-files -z -s \"*.[ch]\" |\n    sed -z -n -e '/^120000/d' -e '/^.* 0\t/s///p' |\n    xargs -r -0 ls -ld\n\nthe result does not look too bad.\n\nOne unrelated tip for better readability is to end, not begin, your\nlines with the \"|\" pipe characters.  That way you do not have to\nsprinkle so many unsightly backslashes ;-).\n\nHTH.\n\n"},{"id":"557738","messageId":"au49oFqexVxPUm-2@debian","threadId":"66791","inReplyTo":"xmqqeccxhg0f.fsf@gitster.g","subject":"Re: git-ls-files(1) avoiding symlinks","fromName":"Alejandro Colomar","fromEmail":"alx+git@kernel.org","sentAt":"2026-11-06T21:06:38Z","receivedAt":"2026-11-06T21:06:38Z","isPatch":false,"sender":{"key":"alx+git@kernel.org","avatar":null},"body":"Hi Junio,\n\n> Date: 2026-11-06 12:16:48-0800\n> From: Junio C Hamano <gitster@pobox.com>\n>\n> Alejandro Colomar <alx+git@kernel.org> writes:\n> \n> >> > With -z it becomes a bit tedious:\n> >> > \n> >> >    $ git ls-files -z --format='%(objectmode) %(path)' \\\n> >> >    | grep -zv ^120000 \\\n> >> >    | sed -z 's/^.* //' \\\n> >> >    | grep -z '\\.[ch]$' \\\n> >> >    | xargs -0 grepc creat;\n> \n> Shouldn't the grep|sed|grep be doable with a single sed invocation?\n> \n> >> > I expect git-ls-files(1) could make this much simpler, though:\n> >> > \n> >> >    $ git ls-files --type f \\\n> >> >    | grep '\\.[ch]$' \\\n> >> >    | xargs grepc creat;\n> \n> Not really.  Why is \"--type\" have to be so special?  What would the\n> next person ask for?  --size?\n\nIndeed.  While looking at the implementation of git-ls-files(1), I\nrealized it wasn't easy to decide how to implement this, as it has other\ntypes (objecttype, which is blob/commit/...).  I thought perhaps\n--objectmode, but then I'm not inderested in searching specific modes,\nbut rather filtering out some.  Perhaps in this case it's more\nappropriate to have my own local command 'git-find' that does what\nI want.\n\nI've ended up doing this, and now don't think a flag in git-ls-files(1)\nwould make much sense.\n\n\t$ cat $(which git-find)\n\t#!/bin/bash\n\t# Copyright 2026, Alejandro Colomar <alx@kernel.org>\n\t# SPDX-License-Identifier: GPL-3.0-or-later\n\n\tset -Eeufo pipefail;\n\n\tgit ls-files --format='%(objectmode) %(path)' \\\n\t| grep -v '^120000' \\\n\t| sed 's/^.* //';\n\nAnd will eventually add a -z flag for this command when I find I need\nit.\n\n> The \"ls-files\" plumbing command is designed as a component to build\n> your own pipeline with.  Don't throw in random unrelated features to\n> it.  In the longer run, it is not a good substitute for improving\n> the skill to construct a good pipeline.\n\nIndeed.\n\n> Learn the tools available to you that you already have heard of, so\n> that you can make better uses of them (in this case \"sed\").  If you\n> squash the grep|sed|grep pipeline into a single sed\n> \n>     git ls-files -z -s \"*.[ch]\" |\n>     sed -z -n -e '/^120000/d' -e '/^.* 0\t/s///p' |\n>     xargs -r -0 ls -ld\n> \n> the result does not look too bad.\n\nI find it simpler to have a simple grep -v rejecting the lines I don't\nwant, and then sed(1) getting rid of the stuff.  The command above seems\ninteresting for performance, but compared to the tool using the file\nnames, that's probably negligible.\n\nAbout git-ls-files(1)'s -s, I think it's easier to parse with the\n%(objectmode) in the format, as I don't need all the other information\nprinted by -s.\n\n> \n> One unrelated tip for better readability is to end, not begin, your\n> lines with the \"|\" pipe characters.  That way you do not have to\n> sprinkle so many unsightly backslashes ;-).\n\nI like to have them at the front of the line because it shows more\nclearly whether I'm piping a command or starting a new pipeline.  I got\nused to the backslashes.  After all, the first few characters of a line\nare all aligned, and the most significant ones.  Stuff to the right I\ntend to ignore it as noise.  :)\n\n> \n> HTH.\n\nYup!  Thanks!\n\n\nHave a lovely night!\nAlex\n\n-- \n<https://www.alejandro-colomar.es>\n"}]}