{"thread":{"id":"38864","subject":"Git ignore help","startedAt":"2015-03-20T10:36:51Z","lastAt":"2015-03-24T20:54:13Z","messageCount":9,"participants":["mdconf@seznam.cz","Eric Sunshine","Duy Nguyen","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"258124","messageId":"47F.3lxhK.7IviRa5Gutr.1L2}X3@seznam.cz","threadId":"38864","inReplyTo":null,"subject":"Git ignore help","fromName":"","fromEmail":"mdconf@seznam.cz","sentAt":"2015-03-20T10:36:51Z","receivedAt":"2015-03-20T10:36:51Z","isPatch":false,"sender":{"key":"mdconf@seznam.cz","avatar":null},"body":"Hello,\nI am trying to setup my git ignore (resp. .git/info/exclude) so that I exclude\n all directories and files except the content of directories that I \nspecifically include (incl. anything within them recursively).\n\nI set the .git/info/exclude with the following content:\n\n========\n# Exclude everything\n/*\n# Except the below that we include\n!/db/data/load/base/bootstraponly\n!/db/data/load/base/safetoload\n!/db/ddl\n!/labels\n!/reports/usrint\n!/scripts\n!/src/cmdsrc/usrint\n========\n\nHowever it does not do what I anticipated. It indeed excludes everything but \nthe include part does not work - it only works for !/labels and !/scripts \ndirectories (i.e. the first level directories). All other are still ignored - \nso when I create file /db/data/load/base/bootstraponly/somefile.txt git still \nignores it...\n\nAny idea what I am doing wrong?\n\nI tried all of the following combinations but still with the same result:\n!/db/data/load/base/bootstraponly\n!/db/data/load/base/bootstraponly/\n!/db/data/load/base/bootstraponly/*\n!/db/data/load/base/bootstraponly/**\n\nWhen I add just \"!/db\" to exclude then it works. But that's not what I want - \nI really want to include just the /db/data/load/base/bootstraponly and its\nall content...\n\nThank you,\nMartin\n"},{"id":"258264","messageId":"CAPig+cRU38sNHRsn=8=60wPO_AoQt-zn1dicFHwjG0OK-9tecQ@mail.gmail.com","threadId":"38864","inReplyTo":"47F.3lxhK.7IviRa5Gutr.1L2}X3@seznam.cz","subject":"Re: Git ignore help","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2015-03-22T03:33:45Z","receivedAt":"2015-03-22T03:33:45Z","isPatch":false,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Mar 20, 2015 at 6:36 AM,  <mdconf@seznam.cz> wrote:\n> I am trying to setup my git ignore (resp. .git/info/exclude) so that I exclude\n>  all directories and files except the content of directories that I\n> specifically include (incl. anything within them recursively).\n>\n> I set the .git/info/exclude with the following content:\n>\n> ========\n> # Exclude everything\n> /*\n> # Except the below that we include\n> !/db/data/load/base/bootstraponly\n> !/db/data/load/base/safetoload\n> !/db/ddl\n> !/labels\n> !/reports/usrint\n> !/scripts\n> !/src/cmdsrc/usrint\n> ========\n>\n> However it does not do what I anticipated. It indeed excludes everything but\n> the include part does not work - it only works for !/labels and !/scripts\n> directories (i.e. the first level directories). All other are still ignored -\n> so when I create file /db/data/load/base/bootstraponly/somefile.txt git still\n> ignores it...\n>\n> Any idea what I am doing wrong?\n\nThe fourth bullet point of the \"Pattern Format\" section of the\ngitignore man page has this to say, which explains the behavior you're\nseeing:\n\n    An optional prefix \"!\" which negates the pattern; any matching\n    file excluded by a previous pattern will become included again.\n    It is not possible to re-include a file if a parent directory of\n    that file is excluded. Git doesn’t list excluded directories for\n    performance reasons, so any patterns on contained files have no\n    effect, no matter where they are defined.\n"},{"id":"258392","messageId":"CACsJy8CGuuRTGycj13cT9d-n=HavzL4LVWC7xjD4Ac=KQGEhBg@mail.gmail.com","threadId":"38864","inReplyTo":"CAPig+cRU38sNHRsn=8=60wPO_AoQt-zn1dicFHwjG0OK-9tecQ@mail.gmail.com","subject":"Re: Git ignore help","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2015-03-24T01:00:23Z","receivedAt":"2015-03-24T01:00:23Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sun, Mar 22, 2015 at 10:33 AM, Eric Sunshine <sunshine@sunshineco.com> wrote:\n> On Fri, Mar 20, 2015 at 6:36 AM,  <mdconf@seznam.cz> wrote:\n>> I am trying to setup my git ignore (resp. .git/info/exclude) so that I exclude\n>>  all directories and files except the content of directories that I\n>> specifically include (incl. anything within them recursively).\n>>\n>> I set the .git/info/exclude with the following content:\n>>\n>> ========\n>> # Exclude everything\n>> /*\n>> # Except the below that we include\n>> !/db/data/load/base/bootstraponly\n>> !/db/data/load/base/safetoload\n>> !/db/ddl\n>> !/labels\n>> !/reports/usrint\n>> !/scripts\n>> !/src/cmdsrc/usrint\n>> ========\n>>\n>> However it does not do what I anticipated. It indeed excludes everything but\n>> the include part does not work - it only works for !/labels and !/scripts\n>> directories (i.e. the first level directories). All other are still ignored -\n>> so when I create file /db/data/load/base/bootstraponly/somefile.txt git still\n>> ignores it...\n>>\n>> Any idea what I am doing wrong?\n>\n> The fourth bullet point of the \"Pattern Format\" section of the\n> gitignore man page has this to say, which explains the behavior you're\n> seeing:\n>\n>     An optional prefix \"!\" which negates the pattern; any matching\n>     file excluded by a previous pattern will become included again.\n>     It is not possible to re-include a file if a parent directory of\n>     that file is excluded. Git doesn’t list excluded directories for\n>     performance reasons, so any patterns on contained files have no\n>     effect, no matter where they are defined.\n\nThis is true. To elaborate, if we have to recurse in excluded\ndirectories so that we can include some back, then the reason for\nexcluding is already defeated as we may need to traverse the entire\ndirectory structure. However in this particular case where we do know\nin advance that only certain directories may have \"re-include\" rules,\ne.g. \"db\", \"reports\" or \"scripts\", we could keep going for a while. I\nthink I attempted to do this in the past and failed (don't remember\nexactly why). Maybe I'll try again some time in future.\n\nAnother option is, if the user is willing to accept performance\ndegradation (in many small repos, it does not matter anyway), then we\ncould keep digging in.\n-- \nDuy\n"},{"id":"258395","messageId":"CAPig+cT5=3kzEu4CzfYW3QtKqDDHn5uSwocoscFLU-T9M-+wiw@mail.gmail.com","threadId":"38864","inReplyTo":"CACsJy8CGuuRTGycj13cT9d-n=HavzL4LVWC7xjD4Ac=KQGEhBg@mail.gmail.com","subject":"Re: Git ignore help","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2015-03-24T01:55:14Z","receivedAt":"2015-03-24T01:55:14Z","isPatch":false,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Mar 23, 2015 at 9:00 PM, Duy Nguyen <pclouds@gmail.com> wrote:\n> On Sun, Mar 22, 2015 at 10:33 AM, Eric Sunshine <sunshine@sunshineco.com> wrote:\n>> On Fri, Mar 20, 2015 at 6:36 AM,  <mdconf@seznam.cz> wrote:\n>>> Any idea what I am doing wrong?\n>>\n>> The fourth bullet point of the \"Pattern Format\" section of the\n>> gitignore man page has this to say, which explains the behavior you're\n>> seeing:\n>>\n>>     An optional prefix \"!\" which negates the pattern; any matching\n>>     file excluded by a previous pattern will become included again.\n>>     It is not possible to re-include a file if a parent directory of\n>>     that file is excluded. Git doesn’t list excluded directories for\n>>     performance reasons, so any patterns on contained files have no\n>>     effect, no matter where they are defined.\n>\n> This is true. To elaborate, if we have to recurse in excluded\n> directories so that we can include some back, then the reason for\n> excluding is already defeated as we may need to traverse the entire\n> directory structure. However in this particular case where we do know\n> in advance that only certain directories may have \"re-include\" rules,\n> e.g. \"db\", \"reports\" or \"scripts\", we could keep going for a while. I\n> think I attempted to do this in the past and failed (don't remember\n> exactly why). Maybe I'll try again some time in future.\n\nI also was pretty sure that you had attempted this, but couldn't find\na reference to it, so I didn't mention it in my response. However,\nwith some more digging, I finally located it[1].\n\n[1]: http://article.gmane.org/gmane.comp.version-control.git/259870\n\n> Another option is, if the user is willing to accept performance\n> degradation (in many small repos, it does not matter anyway), then we\n> could keep digging in.\n> --\n> Duy\n"},{"id":"258402","messageId":"CACsJy8DDC2FbKnu7o1cJR6F6Lh-NabaYtMYOKbm0xbn8PuKoEg@mail.gmail.com","threadId":"38864","inReplyTo":"CAPig+cT5=3kzEu4CzfYW3QtKqDDHn5uSwocoscFLU-T9M-+wiw@mail.gmail.com","subject":"Re: Git ignore help","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2015-03-24T09:39:54Z","receivedAt":"2015-03-24T09:39:54Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Mar 24, 2015 at 8:55 AM, Eric Sunshine <sunshine@sunshineco.com> wrote:\n>> e.g. \"db\", \"reports\" or \"scripts\", we could keep going for a while. I\n>> think I attempted to do this in the past and failed (don't remember\n>> exactly why). Maybe I'll try again some time in future.\n>\n> I also was pretty sure that you had attempted this, but couldn't find\n> a reference to it, so I didn't mention it in my response. However,\n> with some more digging, I finally located it[1].\n>\n> [1]: http://article.gmane.org/gmane.comp.version-control.git/259870\n\nThank you. I only looked at my repo and no branch name suggested it\n(if only there is google search for a git repository..). So I gave up\nbecause of performance reasons again but that was for enabling it\nunconditionaly. If we enable it via a config variable and the user is\nmade aware of the performance implications, I guess it would be ok. So\nit's back in my back log.\n-- \nDuy\n"},{"id":"258405","messageId":"1PE.3lxh4.2cruo5kZ0hh.1L4LoZ@seznam.cz","threadId":"38864","inReplyTo":"CACsJy8DDC2FbKnu7o1cJR6F6Lh-NabaYtMYOKbm0xbn8PuKoEg@mail.gmail.com","subject":"Re: Git ignore help","fromName":"","fromEmail":"mdconf@seznam.cz","sentAt":"2015-03-24T12:46:27Z","receivedAt":"2015-03-24T12:46:27Z","isPatch":false,"sender":{"key":"mdconf@seznam.cz","avatar":null},"body":"Hi Duy, Eric,\n\nthx a lot. So net net - I can't really achieve this. It feels like something very basic and simple so pretty surprising but guess that's what it is :)\n\nNormally, I'd expect that the functionality will behave like with similar other blacklist/whitelist functionalities in Linux - that the more specific definition overrides the less specific one. And when there are 2 on the same level then one (include or exclude) has the priority by default...\n\nAnyways, what I am still not clear on is how come that including just the 'single level folder' work?\n\nDuy, you wrote:\n\n\"This is true. To elaborate, if we have to recurse in excluded directories so that we can include some back, then the reason for excluding is already defeated as we may need to traverse the entire directory structure. However in this particular case where we do know in advance that only certain directories may have \"re-include\" rules, e.g. \"db\", \"reports\" or \"scripts\", we could keep going for a while.\"\n\n... so according to that it sounds like including /db, /reports, /scripts should actually also NOT work. But it does work - i.e. when I add the following:\n\n# exclude\n/*\n\n# except\n!/db\n!/reports\n!/scripts\n\nthen any content within those 3 directories (and their sub directories) is included and not ignored...\n\nIt ONLY does not work when I add more levels - e.g.:\n\n!/reports/something\n\nIn this case neither /reports nor /reports/something or any sub directory is included.\n\nMartin\n\n\n---------- Původní zpráva ----------\nOd: Duy Nguyen \nKomu: Eric Sunshine \nDatum: 24. 3. 2015 10:40:33\nPředmět: Re: Git ignore help\n\nOn Tue, Mar 24, 2015 at 8:55 AM, Eric Sunshine  wrote:\n>> e.g. \"db\", \"reports\" or \"scripts\", we could keep going for a while. I\n>> think I attempted to do this in the past and failed (don't remember\n>> exactly why). Maybe I'll try again some time in future.\n>\n> I also was pretty sure that you had attempted this, but couldn't find\n> a reference to it, so I didn't mention it in my response. However,\n> with some more digging, I finally located it[1].\n>\n> [1]: http://article.gmane.org/gmane.comp.version-control.git/259870\n\nThank you. I only looked at my repo and no branch name suggested it\n(if only there is google search for a git repository..). So I gave up\nbecause of performance reasons again but that was for enabling it\nunconditionaly. If we enable it via a config variable and the user is\nmade aware of the performance implications, I guess it would be ok. So\nit's back in my back log.\n-- \nDuy"},{"id":"258408","messageId":"CACsJy8CY20niCrFYq04KHxS-8xsWdGWcr_vLBvcphcSznbJzFw@mail.gmail.com","threadId":"38864","inReplyTo":"1PE.3lxh4.2cruo5kZ0hh.1L4LoZ@seznam.cz","subject":"Re: Git ignore help","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2015-03-24T13:11:12Z","receivedAt":"2015-03-24T13:11:12Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Mar 24, 2015 at 7:46 PM,  <mdconf@seznam.cz> wrote:\n> Duy, you wrote:\n>\n> \"This is true. To elaborate, if we have to recurse in excluded directories so that we can include some back, then the reason for excluding is already defeated as we may need to traverse the entire directory structure. However in this particular case where we do know in advance that only certain directories may have \"re-include\" rules, e.g. \"db\", \"reports\" or \"scripts\", we could keep going for a while.\"\n>\n> ... so according to that it sounds like including /db, /reports, /scripts should actually also NOT work. But it does work - i.e. when I add the following:\n>\n> # exclude\n> /*\n>\n> # except\n> !/db\n> !/reports\n> !/scripts\n>\n> then any content within those 3 directories (and their sub directories) is included and not ignored...\n>\n> It ONLY does not work when I add more levels - e.g.:\n>\n> !/reports/something\n>\n> In this case neither /reports nor /reports/something or any sub directory is included.\n\nYes. It's the subtlety of optimizing ;-) If you read the man page\nreally carefully (*), \"if the _parent_ directory of that file(**) is\nexcluded\" and the parent of these three directories is _not_ excluded.\n\n(*) I'm not saying this is a good thing. Only docs such as language\nspec or RFCs need that level of attention. But I'm not a good document\nwriter myself, can't blame others. Improvements are welcome though.\n\n(**) \"that file\" should be \"that file or directory\" but I guess\nsimplification here is ok\n-- \nDuy\n"},{"id":"258429","messageId":"CAPig+cQE6-BaOPZQuQJV-ENX+M7oXhozuZmY0LwkdbwDp8xQhw@mail.gmail.com","threadId":"38864","inReplyTo":"CACsJy8DDC2FbKnu7o1cJR6F6Lh-NabaYtMYOKbm0xbn8PuKoEg@mail.gmail.com","subject":"Re: Git ignore help","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2015-03-24T17:37:29Z","receivedAt":"2015-03-24T17:37:29Z","isPatch":false,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Mar 24, 2015 at 5:39 AM, Duy Nguyen <pclouds@gmail.com> wrote:\n> On Tue, Mar 24, 2015 at 8:55 AM, Eric Sunshine <sunshine@sunshineco.com> wrote:\n>>> e.g. \"db\", \"reports\" or \"scripts\", we could keep going for a while. I\n>>> think I attempted to do this in the past and failed (don't remember\n>>> exactly why). Maybe I'll try again some time in future.\n>>\n>> I also was pretty sure that you had attempted this, but couldn't find\n>> a reference to it, so I didn't mention it in my response. However,\n>> with some more digging, I finally located it[1].\n>>\n>> [1]: http://article.gmane.org/gmane.comp.version-control.git/259870\n>\n> Thank you. I only looked at my repo and no branch name suggested it\n> (if only there is google search for a git repository..). So I gave up\n> because of performance reasons again but that was for enabling it\n> unconditionaly. If we enable it via a config variable and the user is\n> made aware of the performance implications, I guess it would be ok. So\n> it's back in my back log.\n\nHow much does a config variable actually help? In a sense, one could\nargue that this is already an opt-in feature since it requires\ncrafting gitignore in a particular fashion. Existing projects which\nhave (properly) functioning gitignore rules won't trigger this\nbehavior. In many cases, Git already allows people to shoot themselves\nin the foot if they desire, thus, as long as the potential performance\nimpact is properly documented, this could be considered another such\ninstance.\n"},{"id":"258449","messageId":"xmqqbnji5d6y.fsf@gitster.dls.corp.google.com","threadId":"38864","inReplyTo":"CAPig+cQE6-BaOPZQuQJV-ENX+M7oXhozuZmY0LwkdbwDp8xQhw@mail.gmail.com","subject":"Re: Git ignore help","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-03-24T20:54:13Z","receivedAt":"2015-03-24T20:54:13Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Sunshine <sunshine@sunshineco.com> writes:\n\n> On Tue, Mar 24, 2015 at 5:39 AM, Duy Nguyen <pclouds@gmail.com> wrote:\n>> On Tue, Mar 24, 2015 at 8:55 AM, Eric Sunshine <sunshine@sunshineco.com> wrote:\n>>>> e.g. \"db\", \"reports\" or \"scripts\", we could keep going for a while. I\n>>>> think I attempted to do this in the past and failed (don't remember\n>>>> exactly why). Maybe I'll try again some time in future.\n>>>\n>>> I also was pretty sure that you had attempted this, but couldn't find\n>>> a reference to it, so I didn't mention it in my response. However,\n>>> with some more digging, I finally located it[1].\n>>>\n>>> [1]: http://article.gmane.org/gmane.comp.version-control.git/259870\n>>\n>> Thank you. I only looked at my repo and no branch name suggested it\n>> (if only there is google search for a git repository..). So I gave up\n>> because of performance reasons again but that was for enabling it\n>> unconditionaly. If we enable it via a config variable and the user is\n>> made aware of the performance implications, I guess it would be ok. So\n>> it's back in my back log.\n>\n> How much does a config variable actually help? In a sense, one could\n> argue that this is already an opt-in feature since it requires\n> crafting gitignore in a particular fashion. Existing projects which\n> have (properly) functioning gitignore rules won't trigger this\n> behavior. In many cases, Git already allows people to shoot themselves\n> in the foot if they desire, thus, as long as the potential performance\n> impact is properly documented, this could be considered another such\n> instance.\n\nYeah, as I re-read that old thread, I really do not think anything\nwrong with the reasoning expressed in the proposed log message.  It\nis pointless to hunt for \"!do-not-exclude-me-please\" in D/.gitignore\nwhen \"D/\" appears in .gitignore in the higher level, but if these two\ni.e.\n\n\tD/\n        !D/do-not-exclude-me-please\n\nappear together in .gitignore in the higher level, we can pay\nattention to that and pick up that single path.  And doing so would\nbe a lot more intuitive to the end user.\n\nMy comment in the thread was only about the documentation being\nunclear and not about the feature, but somehow we failed to\nfollow-up the topic to completion X-<.\n"}]}