{"thread":{"id":"22223","subject":"Gitignore matching \"git add\" wildcard prevents operation","startedAt":"2010-01-14T18:18:40Z","lastAt":"2010-01-15T18:17:26Z","messageCount":12,"participants":["Marko Poutiainen","Jeff King","Junio C Hamano","Nicolas Sebrecht"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"131659","messageId":"4B4F6000.4070005@sofistes.net","threadId":"22223","inReplyTo":null,"subject":"Gitignore matching \"git add\" wildcard prevents operation","fromName":"Marko Poutiainen","fromEmail":"regs@sofistes.net","sentAt":"2010-01-14T18:18:40Z","receivedAt":"2010-01-14T18:18:40Z","isPatch":false,"sender":{"key":"regs@sofistes.net","avatar":null},"body":"It seems that that there is a bug related to how Git handles the add\ncommand with wildcards if the wildcard also matches a file that is ignored.\n\nE.g (I activate the *.[ao] rule when editing .git/info/exclude):\n\nmep@Blackbird:~$ cd /tmp\nmep@Blackbird:/tmp$ mkdir git\nmep@Blackbird:/tmp$ cd git\nmep@Blackbird:/tmp/git$ git init\nInitialized empty Git repository in /tmp/git/.git/\nmep@Blackbird:/tmp/git$ nano -w .git/info/exclude\nmep@Blackbird:/tmp/git$ touch a.o a.c a.h\nmep@Blackbird:/tmp/git$ git add a.*\nThe following paths are ignored by one of your .gitignore files:\na.o\nUse -f if you really want to add them.\nfatal: no files added\nmep@Blackbird:/tmp/git$ git status\n# On branch master\n#\n# Initial commit\n#\n# Untracked files:\n#   (use \"git add <file>...\" to include in what will be committed)\n#\n#       a.c\n#       a.h\n#       a.o\nnothing added to commit but untracked files present (use \"git add\" to track)\n\nFurthermore:\n\nmep@Blackbird:/tmp/git$ git add .\nmep@Blackbird:/tmp/git$ git commit -m \"foo\"\n[master (root-commit) 43da825] foo\n 0 files changed, 0 insertions(+), 0 deletions(-)\n create mode 100644 a.c\n create mode 100644 a.h\nmep@Blackbird:/tmp/git$ nano a.c\nmep@Blackbird:/tmp/git$ nano a.h\nmep@Blackbird:/tmp/git$ git add a.*\nThe following paths are ignored by one of your .gitignore files:\na.o\nUse -f if you really want to add them.\nfatal: no files added\nmep@Blackbird:/tmp/git$ git status\n# On branch master\n# Changed but not updated:\n#   (use \"git add <file>...\" to update what will be committed)\n#   (use \"git checkout -- <file>...\" to discard changes in working\ndirectory)\n#\n#       modified:   a.c\n#       modified:   a.h\n#\nno changes added to commit (use \"git add\" and/or \"git commit -a\")\n"},{"id":"131668","messageId":"20100114195234.GA26684@coredump.intra.peff.net","threadId":"22223","inReplyTo":"4B4F6000.4070005@sofistes.net","subject":"Re: Gitignore matching \"git add\" wildcard prevents operation","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2010-01-14T19:52:34Z","receivedAt":"2010-01-14T19:52:34Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Jan 14, 2010 at 08:18:40PM +0200, Marko Poutiainen wrote:\n\n> It seems that that there is a bug related to how Git handles the add\n> command with wildcards if the wildcard also matches a file that is ignored.\n> \n> E.g (I activate the *.[ao] rule when editing .git/info/exclude):\n> \n> mep@Blackbird:~$ cd /tmp\n> mep@Blackbird:/tmp$ mkdir git\n> mep@Blackbird:/tmp$ cd git\n> mep@Blackbird:/tmp/git$ git init\n> Initialized empty Git repository in /tmp/git/.git/\n> mep@Blackbird:/tmp/git$ nano -w .git/info/exclude\n> mep@Blackbird:/tmp/git$ touch a.o a.c a.h\n> mep@Blackbird:/tmp/git$ git add a.*\n> The following paths are ignored by one of your .gitignore files:\n> a.o\n> Use -f if you really want to add them.\n> fatal: no files added\n\nIt's not a bug. That is working as designed. If \"git add\" encounters a\nproblem with any of the files you gave it on the command line (and\nremember, the shell expands the wildcard, so git literally sees the\nthree files on the command line) then it aborts the entire operation.\n\nNow whether that is useful behavior in this case, I don't know. If you\nwere truly manually specifying the files, then it would probably not be\na big deal to simply remove the offending file. But there is no way to\nuse the shell wildcard and get the behavior you want (not even a \"git\nadd --really-ignore-my-ignores a.*\").\n\nI don't think it would be right to silently ignore them, because it may\nbe the sign of an error. It could be downgraded to a warning, like:\n\n  $ git add *.a\n  warning: a.o is ignored, not adding\n\nThe downside would be that somebody who really _did_ want to add such a\nfile but forgot to use \"-f\" is going to be less likely to notice it, as\nthe command does not barf. So maybe such behavior should be triggered\nwith a command-line option. I dunno.\n\n-Peff\n"},{"id":"131671","messageId":"7vljg0bfox.fsf@alter.siamese.dyndns.org","threadId":"22223","inReplyTo":"20100114195234.GA26684@coredump.intra.peff.net","subject":"Re: Gitignore matching \"git add\" wildcard prevents operation","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-01-14T20:21:34Z","receivedAt":"2010-01-14T20:21:34Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> ... But there is no way to\n> use the shell wildcard and get the behavior you want (not even a \"git\n> add --really-ignore-my-ignores a.*\").\n\nPerhaps you want to run\n\n\t$ git add 'a.*'\n\nNotice that the wildcard is protected from the shell.\n"},{"id":"131672","messageId":"20100114203928.GA26883@coredump.intra.peff.net","threadId":"22223","inReplyTo":"7vljg0bfox.fsf@alter.siamese.dyndns.org","subject":"Re: Gitignore matching \"git add\" wildcard prevents operation","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2010-01-14T20:39:28Z","receivedAt":"2010-01-14T20:39:28Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Jan 14, 2010 at 12:21:34PM -0800, Junio C Hamano wrote:\n\n> Jeff King <peff@peff.net> writes:\n> \n> > ... But there is no way to\n> > use the shell wildcard and get the behavior you want (not even a \"git\n> > add --really-ignore-my-ignores a.*\").\n> \n> Perhaps you want to run\n> \n> \t$ git add 'a.*'\n> \n> Notice that the wildcard is protected from the shell.\n\nUgh. You're right that it does work, but I don't expect users to make\nthe intuitive jump from the OP's problem to this solution (I certainly\ndidn't). In particular:\n\n  1. Most programs don't take their own globs. Without knowing that git\n     can do so, there is no reason to discover it in this instance. I\n     can see searching the manpage for options, but not for a discussion\n     of globbing behavior.\n\n  2. They would have to know that using a git-glob will magically change\n     the error-checking behavior.\n\n-Peff\n"},{"id":"131676","messageId":"4B4F87A0.2090301@sofistes.net","threadId":"22223","inReplyTo":"20100114195234.GA26684@coredump.intra.peff.net","subject":"Re: Gitignore matching \"git add\" wildcard prevents operation","fromName":"Marko Poutiainen","fromEmail":"regs@sofistes.net","sentAt":"2010-01-14T21:07:44Z","receivedAt":"2010-01-14T21:07:44Z","isPatch":false,"sender":{"key":"regs@sofistes.net","avatar":null},"body":"Jeff King wrote:\n> On Thu, Jan 14, 2010 at 08:18:40PM +0200, Marko Poutiainen wrote:\n> \n>> It seems that that there is a bug related to how Git handles the add\n>> command with wildcards if the wildcard also matches a file that is ignored.\n>>\n>> E.g (I activate the *.[ao] rule when editing .git/info/exclude):\n>>\n>> mep@Blackbird:~$ cd /tmp\n>> mep@Blackbird:/tmp$ mkdir git\n>> mep@Blackbird:/tmp$ cd git\n>> mep@Blackbird:/tmp/git$ git init\n>> Initialized empty Git repository in /tmp/git/.git/\n>> mep@Blackbird:/tmp/git$ nano -w .git/info/exclude\n>> mep@Blackbird:/tmp/git$ touch a.o a.c a.h\n>> mep@Blackbird:/tmp/git$ git add a.*\n>> The following paths are ignored by one of your .gitignore files:\n>> a.o\n>> Use -f if you really want to add them.\n>> fatal: no files added\n> \n> It's not a bug. That is working as designed. If \"git add\" encounters a\n> problem with any of the files you gave it on the command line (and\n> remember, the shell expands the wildcard, so git literally sees the\n> three files on the command line) then it aborts the entire operation.\n> \n> Now whether that is useful behavior in this case, I don't know. If you\n> were truly manually specifying the files, then it would probably not be\n> a big deal to simply remove the offending file. But there is no way to\n> use the shell wildcard and get the behavior you want (not even a \"git\n> add --really-ignore-my-ignores a.*\").\n> \n> I don't think it would be right to silently ignore them, because it may\n> be the sign of an error. It could be downgraded to a warning, like:\n> \n>   $ git add *.a\n>   warning: a.o is ignored, not adding\n> \n> The downside would be that somebody who really _did_ want to add such a\n> file but forgot to use \"-f\" is going to be less likely to notice it, as\n> the command does not barf. So maybe such behavior should be triggered\n> with a command-line option. I dunno.\n\nWell, it then the interface just isn't consistent, because if that's the\nreason, then why does\n\n$git add .\n\nthen work in this case? . is still just a type of wildcard, so it fail\nin a similar way, shouldn't it?\n\nYour idea of giving a warning is good as is the command-line option. In\nany case, in my opinion this feature can make some operations more\ncomplex than they should be. For instance, in my example I would have to\nseparately do git add to both files. What if there are more than two\nfiles to add and I get this error every time I try to add the files?\n"},{"id":"131725","messageId":"20100115153419.GA12982@vidovic","threadId":"22223","inReplyTo":"20100114203928.GA26883@coredump.intra.peff.net","subject":"Re: Gitignore matching \"git add\" wildcard prevents operation","fromName":"Nicolas Sebrecht","fromEmail":"nicolas.s.dev@gmx.fr","sentAt":"2010-01-15T15:34:19Z","receivedAt":"2010-01-15T15:34:19Z","isPatch":false,"sender":{"key":"nicolas.s.dev@gmx.fr","avatar":null},"body":"The 14/01/10, Jeff King wrote:\n\n>   1. Most programs don't take their own globs. Without knowing that git\n>      can do so, there is no reason to discover it in this instance. I\n>      can see searching the manpage for options, but not for a discussion\n>      of globbing behavior.\n> \n>   2. They would have to know that using a git-glob will magically change\n>      the error-checking behavior.\n\nNot sure. This isn't a Git-particular issue.\n\nUsers may hit this with a lot of other unix tools (sed, grep, find,\netc). So, we can expect either\n  they already know the issue;\nor\n  they are discovering it using Git.\n\nMost of the tools I talk about do have a manual section about globbing.\nUsers could learn globs with Git too and expect the same behaviour\nsomewhere else.\n\n\n-- \nNicolas Sebrecht\n"},{"id":"131726","messageId":"20100115153936.GB12982@vidovic","threadId":"22223","inReplyTo":"4B4F87A0.2090301@sofistes.net","subject":"Re: Gitignore matching \"git add\" wildcard prevents operation","fromName":"Nicolas Sebrecht","fromEmail":"nicolas.s.dev@gmx.fr","sentAt":"2010-01-15T15:39:36Z","receivedAt":"2010-01-15T15:39:36Z","isPatch":false,"sender":{"key":"nicolas.s.dev@gmx.fr","avatar":null},"body":"The 14/01/10, Marko Poutiainen wrote:\n> \n> Well, it then the interface just isn't consistent,\n\nBut it is...\n\n> Well, it then the interface just isn't consistent, because if that's the\n> reason, then why does\n> \n> $git add .\n> \n> then work in this case? . is still just a type of wildcard, so it fail\n> in a similar way, shouldn't it?\n\n...because the dot isn't a wildcard. It's a path.\n\nSo, we expect 'git add' to recusively work and resolve ignored paths\nfrom .gitignore and so.\n\n-- \nNicolas Sebrecht\n"},{"id":"131729","messageId":"20100115154816.GA1579@coredump.intra.peff.net","threadId":"22223","inReplyTo":"20100115153419.GA12982@vidovic","subject":"Re: Gitignore matching \"git add\" wildcard prevents operation","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2010-01-15T15:48:16Z","receivedAt":"2010-01-15T15:48:16Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Jan 15, 2010 at 04:34:19PM +0100, Nicolas Sebrecht wrote:\n\n> >   1. Most programs don't take their own globs. Without knowing that git\n> >      can do so, there is no reason to discover it in this instance. I\n> >      can see searching the manpage for options, but not for a discussion\n> >      of globbing behavior.\n> > \n> >   2. They would have to know that using a git-glob will magically change\n> >      the error-checking behavior.\n> \n> Not sure. This isn't a Git-particular issue.\n> \n> Users may hit this with a lot of other unix tools (sed, grep, find,\n> etc). So, we can expect either\n>   they already know the issue;\n> or\n>   they are discovering it using Git.\n\nI don't understand what you mean. How does \"sed\" do its own globbing of\nthe command line?\n\n> Most of the tools I talk about do have a manual section about globbing.\n> Users could learn globs with Git too and expect the same behaviour\n> somewhere else.\n\nSure. git-add(1) talks about globbing, too, and it even has a sentence\nthat explains exactly what is happening in Marko's case:\n\n   The git add command will not add ignored files by default. If any\n   ignored files were explicitly specified on the command line, git add\n   will fail with a list of ignored files. Ignored files reached by\n   directory recursion or filename globbing performed by Git (quote your\n   globs before the shell) will be silently ignored.\n\nMy complaint was more that as a user, I am not likely to see this\nproblem and think \"I'll bet git-specific globbing can solve it.\" And\nwhen I look in the manual, I am more likely to look for a command-line\noption that helps me rather than to read all of the text (though the\nfact that we specifically mention \"ignore\" in the same paragraph means\nthat at least a user searching for \"ignore\" in the man page will come up\nwith this paragraph).\n\n-Peff\n"},{"id":"131730","messageId":"20100115161107.GC12982@vidovic","threadId":"22223","inReplyTo":"20100115154816.GA1579@coredump.intra.peff.net","subject":"Re: Gitignore matching \"git add\" wildcard prevents operation","fromName":"Nicolas Sebrecht","fromEmail":"nicolas.s.dev@gmx.fr","sentAt":"2010-01-15T16:11:07Z","receivedAt":"2010-01-15T16:11:07Z","isPatch":false,"sender":{"key":"nicolas.s.dev@gmx.fr","avatar":null},"body":"The 15/01/10, Jeff King wrote:\n> On Fri, Jan 15, 2010 at 04:34:19PM +0100, Nicolas Sebrecht wrote:\n> \n> > >   1. Most programs don't take their own globs. Without knowing that git\n> > >      can do so, there is no reason to discover it in this instance. I\n> > >      can see searching the manpage for options, but not for a discussion\n> > >      of globbing behavior.\n> > > \n> > >   2. They would have to know that using a git-glob will magically change\n> > >      the error-checking behavior.\n> > \n> > Not sure. This isn't a Git-particular issue.\n> > \n> > Users may hit this with a lot of other unix tools (sed, grep, find,\n> > etc). So, we can expect either\n> >   they already know the issue;\n> > or\n> >   they are discovering it using Git.\n> \n> I don't understand what you mean. How does \"sed\" do its own globbing of\n> the command line?\n\nWell, we are in the same dilemma as the other tools. The internal\nglobbing rules are explained in the related man page.\n\n> > Most of the tools I talk about do have a manual section about globbing.\n> > Users could learn globs with Git too and expect the same behaviour\n> > somewhere else.\n> \n> Sure. git-add(1) talks about globbing, too\n\nOops, I was missing that; thanks.\n\n> My complaint was more that as a user, I am not likely to see this\n> problem and think \"I'll bet git-specific globbing can solve it.\"\n\nYes. My point is that we are not talking about a Git specific issue.\nWhat you're raising here is true whatever the command is. So, as long as\nwe clearly explain how 'git add' works, we are fine. Don't we?\n\n>                                                                  And\n> when I look in the manual, I am more likely to look for a command-line\n> option that helps me rather than to read all of the text\n\nTrue. All I can see is to improve the man page with a dedicated section\n\"Globbing\" instead of loosing it in a \"random\" place.\n\n-- \nNicolas Sebrecht\n"},{"id":"131731","messageId":"20100115163021.GA1947@coredump.intra.peff.net","threadId":"22223","inReplyTo":"20100115161107.GC12982@vidovic","subject":"Re: Gitignore matching \"git add\" wildcard prevents operation","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2010-01-15T16:30:21Z","receivedAt":"2010-01-15T16:30:21Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Jan 15, 2010 at 05:11:07PM +0100, Nicolas Sebrecht wrote:\n\n> > I don't understand what you mean. How does \"sed\" do its own globbing of\n> > the command line?\n> \n> Well, we are in the same dilemma as the other tools. The internal\n> globbing rules are explained in the related man page.\n\nMaybe I wasn't clear: to my knowledge, \"sed\" does not do any globbing\nitself.  How is this the same situation?\n\nOf course other commands like sed will be fed the expansion of a shell\nglob, and there may be times when you want to feed a subset of an\nexpansion. But that is not my complaint; my complaint was mainly that\ngit's solution to this is not easily discoverable by an uninformed user.\nMost other commands don't even have a solution (you would have to solve\nit in the shell to pass the desired expansion to the program).\n\n> > when I look in the manual, I am more likely to look for a command-line\n> > option that helps me rather than to read all of the text\n> \n> True. All I can see is to improve the man page with a dedicated section\n> \"Globbing\" instead of loosing it in a \"random\" place.\n\nI don't think that would help. The problem is that the user knows they\nhave an issue with ignored files. The solution is custom globbing, but\nthey don't know that. So making globbing more prominent doesn't help,\nsince they will be looking for ignores. You would need to have an\n\"ignore\" section that mentions globbing.\n\nTo be clear: I do not have an actual solution, and my initial message\nwas mostly just grumbling. We _do_ mention globbing and ignores in the\nsame paragraph, as I quoted earlier. So that is probably enough for a\ndiligent user to come up with the solution, or at least enough that\ntrying to improve on it will have diminishing returns.\n\nYou could even argue that I was not being such a diligent user in my\ninitial response. :)\n\n-Peff\n"},{"id":"131740","messageId":"20100115180612.GD12982@vidovic","threadId":"22223","inReplyTo":"20100115163021.GA1947@coredump.intra.peff.net","subject":"Re: Gitignore matching \"git add\" wildcard prevents operation","fromName":"Nicolas Sebrecht","fromEmail":"nicolas.s.dev@gmx.fr","sentAt":"2010-01-15T18:06:12Z","receivedAt":"2010-01-15T18:06:12Z","isPatch":false,"sender":{"key":"nicolas.s.dev@gmx.fr","avatar":null},"body":"The 15/01/10, Jeff King wrote:\n> On Fri, Jan 15, 2010 at 05:11:07PM +0100, Nicolas Sebrecht wrote:\n> \n> > > I don't understand what you mean. How does \"sed\" do its own globbing of\n> > > the command line?\n> > \n> > Well, we are in the same dilemma as the other tools. The internal\n> > globbing rules are explained in the related man page.\n> \n> Maybe I wasn't clear: to my knowledge, \"sed\" does not do any globbing\n> itself.\n\n\"sed\" does /pattern matching/ which is the meaning I give to /globbing/.\n\nSure, \"sed\" doesn't do /shell globbing/ but is fed from it. We obviously\nagree here. :-)\n\n>            But that is not my complaint; my complaint was mainly that\n> git's solution to this is not easily discoverable by an uninformed user.\n\nI agree in that it is not easily discoverable. But I don't think Git's\nsolution is worse than any other tool. The \"discoverabiliy\" looks equal\nto any other tool for me. As you said, to do a good job we should be\nable to know what was the original command-line, which isn't possible.\n\n\n-- \nNicolas Sebrecht\n"},{"id":"131743","messageId":"7v7hrj1bd5.fsf@alter.siamese.dyndns.org","threadId":"22223","inReplyTo":"20100115161107.GC12982@vidovic","subject":"Re: Gitignore matching \"git add\" wildcard prevents operation","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-01-15T18:17:26Z","receivedAt":"2010-01-15T18:17:26Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nicolas Sebrecht <nicolas.s.dev@gmx.fr> writes:\n\n>> My complaint was more that as a user, I am not likely to see this\n>> problem and think \"I'll bet git-specific globbing can solve it.\"\n>\n> Yes. My point is that we are not talking about a Git specific issue.\n\nActually it is.  The thing is, git subcommands do *not* take individual\nfilenames in general from their command line.  They _always_ take pathspecs,\nfind files in the work tree that match the pathspecs (but excludes ignored\nones), and work on them [*1*].\n\nOther commands usually don't do that.\n\nSo \"git add .\" and \"git add 'a.*'\" behave exactly the same way.  Both find\nfiles that match their respective pathspecs.  The former knows that \".\"\nmatches \"all files that exist in the cwd (excluding ignored ones)\" and the\nlatter knows that \"a.*\" matches \"all files that fnmatch(3) 'a.*' glob\n(excluding ignored ones)\".  Hence you will not get \"don't try to add a\nfile that you said you want to ignore; use -f if you mean it\".\n\n\n[Footnote]\n\n*1* There are exceptions, such as parameters to the \"--file\" option, and\n\"git blame\".\n"}]}