{"thread":{"id":"38951","subject":"git ls-files wildcard behavior considered harmful","startedAt":"2015-03-30T23:04:59Z","lastAt":"2015-03-31T00:35:24Z","messageCount":6,"participants":["Joey Hess","Duy Nguyen","Jonathan Nieder","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"258704","messageId":"20150330230459.GA13927@kitenet.net","threadId":"38951","inReplyTo":null,"subject":"git ls-files wildcard behavior considered harmful","fromName":"Joey Hess","fromEmail":"id@joeyh.name","sentAt":"2015-03-30T23:04:59Z","receivedAt":"2015-03-30T23:04:59Z","isPatch":false,"sender":{"key":"id@joeyh.name","avatar":"https://avatars.githubusercontent.com/u/16392?v=4"},"body":"I was very surprised to learn the other day that git ls-files 'foo*'\nwill expand wildcards (including character classes), in the absence of\nexpansion by the shell. (git version 2.1.4)\n\njoey@darkstar:~/tmp/aaa>git ls-files 'foo*bar'\nfoo*bar\nfoobazbar\njoey@darkstar:~/tmp/aaa>git ls-files '[abc]'\n[abc]\na\nb\n\nAs far as I can see this behavior is not documented on the man page,\nexcept for a tiny mention in the --with-tree documentation, where\nit says \"<file> (i.e. path pattern)\".\n\nSince I wanted to avoid this wildcard expension, I tried slash-escaping\nthe wildcard characters. This works:\n\njoey@darkstar:~/tmp/aaa>git ls-files 'foo\\*bar'\nfoo*bar\njoey@darkstar:~/tmp/aaa>git ls-files '\\[abc\\]'\n[abc]\n\nBut, there is a weird behavior here with subdirectories. While normally\nls-files would recurse, slash-escaped wildcard characters in the directory\nname prevent recursion.\n\njoey@darkstar:~/tmp/aaa>git ls-files 'foo[d]'\nfoo[d]/subfile\nfood\njoey@darkstar:~/tmp/aaa>git ls-files 'foo\\[d\\]'\njoey@darkstar:~/tmp/aaa>\n\nThe above example shows a case where it's impossible to get ls-files\nto only list files in a directory and not other files that match the\nwildcard. This seems like it must be a bug, and it means it's impossible\nto reliably work around the wildcard expansion behavior.\n\nI suspect that this wildcard expansion behavior is useful somewhere.\nBut from the perspective of using ls-files as plumbing, where you want\nto get out some subset of what was put in, it's not nice.\n\n-- \nsee shy jo\n"},{"id":"258705","messageId":"CACsJy8BDoXHS6nj7pEe0i8T=bMD8aG5FwuXABKgPOQBeMOr+Vw@mail.gmail.com","threadId":"38951","inReplyTo":"20150330230459.GA13927@kitenet.net","subject":"Re: git ls-files wildcard behavior considered harmful","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2015-03-30T23:16:52Z","receivedAt":"2015-03-30T23:16:52Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Mar 31, 2015 at 6:04 AM, Joey Hess <id@joeyh.name> wrote:\n> I was very surprised to learn the other day that git ls-files 'foo*'\n> will expand wildcards (including character classes), in the absence of\n> expansion by the shell. (git version 2.1.4)\n>\n> joey@darkstar:~/tmp/aaa>git ls-files 'foo*bar'\n> foo*bar\n> foobazbar\n> joey@darkstar:~/tmp/aaa>git ls-files '[abc]'\n> [abc]\n> a\n> b\n>\n> As far as I can see this behavior is not documented on the man page,\n> except for a tiny mention in the --with-tree documentation, where\n> it says \"<file> (i.e. path pattern)\".\n>\n> Since I wanted to avoid this wildcard expension, I tried slash-escaping\n> the wildcard characters.\n\nYou can do \"git --literal-pathspecs ls-files ...\" or set GIT_LITERAL_PATHSPECS.\n-- \nDuy\n"},{"id":"258706","messageId":"20150330232424.GD22844@google.com","threadId":"38951","inReplyTo":"20150330230459.GA13927@kitenet.net","subject":"Re: git ls-files wildcard behavior considered harmful","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2015-03-30T23:24:24Z","receivedAt":"2015-03-30T23:24:24Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi Joey,\n\nJoey Hess wrote:\n\n> Since I wanted to avoid this wildcard expension, I tried slash-escaping\n> the wildcard characters. This works:\n>\n> joey@darkstar:~/tmp/aaa>git ls-files 'foo\\*bar'\n> foo*bar\n\nDoes 'git --noglob-pathspecs' help?\n\nCurious,\nJonathan\n"},{"id":"258708","messageId":"20150330233614.GA17089@kitenet.net","threadId":"38951","inReplyTo":"CACsJy8BDoXHS6nj7pEe0i8T=bMD8aG5FwuXABKgPOQBeMOr+Vw@mail.gmail.com","subject":"Re: git ls-files wildcard behavior considered harmful","fromName":"Joey Hess","fromEmail":"id@joeyh.name","sentAt":"2015-03-30T23:36:14Z","receivedAt":"2015-03-30T23:36:14Z","isPatch":false,"sender":{"key":"id@joeyh.name","avatar":"https://avatars.githubusercontent.com/u/16392?v=4"},"body":"Duy Nguyen wrote:\n> You can do \"git --literal-pathspecs ls-files ...\" or set GIT_LITERAL_PATHSPECS.\n\nThanks! --literal-pathspecs does allow getting around this.\n\nNow I'm wondering what other parts of plumbing might be doing globbing\nthat I did not anticipate. Maybe I should set the environment variable\nso I don't need to worry about it..\n\n-- \nsee shy jo\n"},{"id":"258710","messageId":"20150331000924.GA27090@peff.net","threadId":"38951","inReplyTo":"20150330233614.GA17089@kitenet.net","subject":"Re: git ls-files wildcard behavior considered harmful","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2015-03-31T00:09:24Z","receivedAt":"2015-03-31T00:09:24Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Mar 30, 2015 at 07:36:14PM -0400, Joey Hess wrote:\n\n> Duy Nguyen wrote:\n> > You can do \"git --literal-pathspecs ls-files ...\" or set GIT_LITERAL_PATHSPECS.\n> \n> Thanks! --literal-pathspecs does allow getting around this.\n> \n> Now I'm wondering what other parts of plumbing might be doing globbing\n> that I did not anticipate. Maybe I should set the environment variable\n> so I don't need to worry about it..\n\nPretty much everything takes magic patterns[1]. Pathspecs given to diff,\npath limiters for pruning, etc. If you are scripting with raw filenames\n(e.g., feeding the filenames out of another tool), I recommend setting\nGIT_LITERAL_PATHSPECS everywhere. That's what we do on the server side\nat GitHub (and is the reason I implemented --literal-pathspecs in the\nfirst place).\n\n-Peff\n\n[1] Note that globbing is just part of this. Names starting with colon\n    are also magical, e.g. \":/foo\", or even \":(literal)foo\".  These are\n    documented in the \"pathspec\" definition of \"git help glossary\".\n"},{"id":"258711","messageId":"CACsJy8C9VdpN2MoNC7-ZuLAvjV5fUZVLzMM-gUzjj1ZQ5r9Vtg@mail.gmail.com","threadId":"38951","inReplyTo":"20150330233614.GA17089@kitenet.net","subject":"Re: git ls-files wildcard behavior considered harmful","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2015-03-31T00:35:24Z","receivedAt":"2015-03-31T00:35:24Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Mar 31, 2015 at 6:36 AM, Joey Hess <id@joeyh.name> wrote:\n> Duy Nguyen wrote:\n>> You can do \"git --literal-pathspecs ls-files ...\" or set GIT_LITERAL_PATHSPECS.\n>\n> Thanks! --literal-pathspecs does allow getting around this.\n>\n> Now I'm wondering what other parts of plumbing might be doing globbing\n> that I did not anticipate. Maybe I should set the environment variable\n> so I don't need to worry about it..\n\nWe're just a bit sloppy in documentation. ls-files learned about\nwildcards since 56fc510 ([PATCH] git-ls-files: generalized pathspecs -\n2005-08-21). Two months later, git-ls-files.txt is updated to say\n\"<file>\" (which is not entirely correct even if we ignore wildcards,\nls-files can take directories as well). With latest git, I think only\ngit-mv and git-read-tree go with plain paths. It's probably best to\nset GIT_LITERAL_PATHSPECS for scripting like Jeff said.\n-- \nDuy\n"}]}