# git ls-files wildcard behavior considered harmful 6 messages from 2015-03-30 to 2015-03-31. Participants: Joey Hess, Duy Nguyen, Jonathan Nieder, Jeff King. Thread: https://gitlist.dev/t/38951 ## Joey Hess, 2015-03-30 23:04 Subject: git ls-files wildcard behavior considered harmful Message-ID: <20150330230459.GA13927@kitenet.net> URL: https://gitlist.dev/e/20150330230459.GA13927%40kitenet.net ``` I was very surprised to learn the other day that git ls-files 'foo*' will expand wildcards (including character classes), in the absence of expansion by the shell. (git version 2.1.4) joey@darkstar:~/tmp/aaa>git ls-files 'foo*bar' foo*bar foobazbar joey@darkstar:~/tmp/aaa>git ls-files '[abc]' [abc] a b As far as I can see this behavior is not documented on the man page, except for a tiny mention in the --with-tree documentation, where it says " (i.e. path pattern)". Since I wanted to avoid this wildcard expension, I tried slash-escaping the wildcard characters. This works: joey@darkstar:~/tmp/aaa>git ls-files 'foo\*bar' foo*bar joey@darkstar:~/tmp/aaa>git ls-files '\[abc\]' [abc] But, there is a weird behavior here with subdirectories. While normally ls-files would recurse, slash-escaped wildcard characters in the directory name prevent recursion. joey@darkstar:~/tmp/aaa>git ls-files 'foo[d]' foo[d]/subfile food joey@darkstar:~/tmp/aaa>git ls-files 'foo\[d\]' joey@darkstar:~/tmp/aaa> The above example shows a case where it's impossible to get ls-files to only list files in a directory and not other files that match the wildcard. This seems like it must be a bug, and it means it's impossible to reliably work around the wildcard expansion behavior. I suspect that this wildcard expansion behavior is useful somewhere. But from the perspective of using ls-files as plumbing, where you want to get out some subset of what was put in, it's not nice. -- see shy jo ``` ## Duy Nguyen, 2015-03-30 23:16 Subject: Re: git ls-files wildcard behavior considered harmful Message-ID: URL: https://gitlist.dev/e/CACsJy8BDoXHS6nj7pEe0i8T%3DbMD8aG5FwuXABKgPOQBeMOr%2BVw%40mail.gmail.com In-Reply-To: <20150330230459.GA13927@kitenet.net> ``` On Tue, Mar 31, 2015 at 6:04 AM, Joey Hess wrote: > I was very surprised to learn the other day that git ls-files 'foo*' > will expand wildcards (including character classes), in the absence of > expansion by the shell. (git version 2.1.4) > > joey@darkstar:~/tmp/aaa>git ls-files 'foo*bar' > foo*bar > foobazbar > joey@darkstar:~/tmp/aaa>git ls-files '[abc]' > [abc] > a > b > > As far as I can see this behavior is not documented on the man page, > except for a tiny mention in the --with-tree documentation, where > it says " (i.e. path pattern)". > > Since I wanted to avoid this wildcard expension, I tried slash-escaping > the wildcard characters. You can do "git --literal-pathspecs ls-files ..." or set GIT_LITERAL_PATHSPECS. -- Duy ``` ## Jonathan Nieder, 2015-03-30 23:24 Subject: Re: git ls-files wildcard behavior considered harmful Message-ID: <20150330232424.GD22844@google.com> URL: https://gitlist.dev/e/20150330232424.GD22844%40google.com In-Reply-To: <20150330230459.GA13927@kitenet.net> ``` Hi Joey, Joey Hess wrote: > Since I wanted to avoid this wildcard expension, I tried slash-escaping > the wildcard characters. This works: > > joey@darkstar:~/tmp/aaa>git ls-files 'foo\*bar' > foo*bar Does 'git --noglob-pathspecs' help? Curious, Jonathan ``` ## Joey Hess, 2015-03-30 23:36 Subject: Re: git ls-files wildcard behavior considered harmful Message-ID: <20150330233614.GA17089@kitenet.net> URL: https://gitlist.dev/e/20150330233614.GA17089%40kitenet.net In-Reply-To: ``` Duy Nguyen wrote: > You can do "git --literal-pathspecs ls-files ..." or set GIT_LITERAL_PATHSPECS. Thanks! --literal-pathspecs does allow getting around this. Now I'm wondering what other parts of plumbing might be doing globbing that I did not anticipate. Maybe I should set the environment variable so I don't need to worry about it.. -- see shy jo ``` ## Jeff King, 2015-03-31 00:09 Subject: Re: git ls-files wildcard behavior considered harmful Message-ID: <20150331000924.GA27090@peff.net> URL: https://gitlist.dev/e/20150331000924.GA27090%40peff.net In-Reply-To: <20150330233614.GA17089@kitenet.net> ``` On Mon, Mar 30, 2015 at 07:36:14PM -0400, Joey Hess wrote: > Duy Nguyen wrote: > > You can do "git --literal-pathspecs ls-files ..." or set GIT_LITERAL_PATHSPECS. > > Thanks! --literal-pathspecs does allow getting around this. > > Now I'm wondering what other parts of plumbing might be doing globbing > that I did not anticipate. Maybe I should set the environment variable > so I don't need to worry about it.. Pretty much everything takes magic patterns[1]. Pathspecs given to diff, path limiters for pruning, etc. If you are scripting with raw filenames (e.g., feeding the filenames out of another tool), I recommend setting GIT_LITERAL_PATHSPECS everywhere. That's what we do on the server side at GitHub (and is the reason I implemented --literal-pathspecs in the first place). -Peff [1] Note that globbing is just part of this. Names starting with colon are also magical, e.g. ":/foo", or even ":(literal)foo". These are documented in the "pathspec" definition of "git help glossary". ``` ## Duy Nguyen, 2015-03-31 00:35 Subject: Re: git ls-files wildcard behavior considered harmful Message-ID: URL: https://gitlist.dev/e/CACsJy8C9VdpN2MoNC7-ZuLAvjV5fUZVLzMM-gUzjj1ZQ5r9Vtg%40mail.gmail.com In-Reply-To: <20150330233614.GA17089@kitenet.net> ``` On Tue, Mar 31, 2015 at 6:36 AM, Joey Hess wrote: > Duy Nguyen wrote: >> You can do "git --literal-pathspecs ls-files ..." or set GIT_LITERAL_PATHSPECS. > > Thanks! --literal-pathspecs does allow getting around this. > > Now I'm wondering what other parts of plumbing might be doing globbing > that I did not anticipate. Maybe I should set the environment variable > so I don't need to worry about it.. We're just a bit sloppy in documentation. ls-files learned about wildcards since 56fc510 ([PATCH] git-ls-files: generalized pathspecs - 2005-08-21). Two months later, git-ls-files.txt is updated to say "" (which is not entirely correct even if we ignore wildcards, ls-files can take directories as well). With latest git, I think only git-mv and git-read-tree go with plain paths. It's probably best to set GIT_LITERAL_PATHSPECS for scripting like Jeff said. -- Duy ```