threads / discuss / 38864

Git ignore help

Subject: Git ignore help

## tl;dr

9 messages between Mar 20, 2015 and Mar 24, 2015.

replies: 8people: 4as markdown or json

mdconf@seznam.cz· Mar 20, 2015, 10:36 UTC · lore
Hello,
I am trying to setup my git ignore (resp. .git/info/exclude) so that I exclude
 all directories and files except the content of directories that I 
specifically include (incl. anything within them recursively).
I set the .git/info/exclude with the following content:

======== # Exclude everything /* # Except the below that we include !/db/data/load/base/bootstraponly !/db/data/load/base/safetoload !/db/ddl !/labels !/reports/usrint !/scripts !/src/cmdsrc/usrint ========

However it does not do what I anticipated. It indeed excludes everything but the include part does not work - it only works for !/labels and !/scripts directories (i.e. the first level directories). All other are still ignored - so when I create file /db/data/load/base/bootstraponly/somefile.txt git still ignores it...

Any idea what I am doing wrong?

I tried all of the following combinations but still with the same result: !/db/data/load/base/bootstraponly !/db/data/load/base/bootstraponly/ !/db/data/load/base/bootstraponly/* !/db/data/load/base/bootstraponly/**

When I add just "!/db" to exclude then it works. But that's not what I want - I really want to include just the /db/data/load/base/bootstraponly and its all content...

Thank you, Martin

Eric Sunshine· Mar 22, 2015, 03:33 UTC · re: mdconf@seznam.cz · lore

Re: Git ignore help

On Fri, Mar 20, 2015 at 6:36 AM,  <mdconf@seznam.cz> wrote:
Show 26 quoted lines
> I am trying to setup my git ignore (resp. .git/info/exclude) so that I exclude
>  all directories and files except the content of directories that I
> specifically include (incl. anything within them recursively).
>
> I set the .git/info/exclude with the following content:
>
> ========
> # Exclude everything
> /*
> # Except the below that we include
> !/db/data/load/base/bootstraponly
> !/db/data/load/base/safetoload
> !/db/ddl
> !/labels
> !/reports/usrint
> !/scripts
> !/src/cmdsrc/usrint
> ========
>
> However it does not do what I anticipated. It indeed excludes everything but
> the include part does not work - it only works for !/labels and !/scripts
> directories (i.e. the first level directories). All other are still ignored -
> so when I create file /db/data/load/base/bootstraponly/somefile.txt git still
> ignores it...
>
> Any idea what I am doing wrong?

The fourth bullet point of the "Pattern Format" section of the gitignore man page has this to say, which explains the behavior you're seeing:

    An optional prefix "!" which negates the pattern; any matching
    file excluded by a previous pattern will become included again.
    It is not possible to re-include a file if a parent directory of
    that file is excluded. Git doesn’t list excluded directories for
    performance reasons, so any patterns on contained files have no
    effect, no matter where they are defined.
Duy Nguyen· Mar 24, 2015, 01:00 UTC · re: Eric Sunshine · lore

Re: Git ignore help

On Sun, Mar 22, 2015 at 10:33 AM, Eric Sunshine <sunshine@sunshineco.com> wrote:
Show 38 quoted lines
> On Fri, Mar 20, 2015 at 6:36 AM,  <mdconf@seznam.cz> wrote:
>> I am trying to setup my git ignore (resp. .git/info/exclude) so that I exclude
>>  all directories and files except the content of directories that I
>> specifically include (incl. anything within them recursively).
>>
>> I set the .git/info/exclude with the following content:
>>
>> ========
>> # Exclude everything
>> /*
>> # Except the below that we include
>> !/db/data/load/base/bootstraponly
>> !/db/data/load/base/safetoload
>> !/db/ddl
>> !/labels
>> !/reports/usrint
>> !/scripts
>> !/src/cmdsrc/usrint
>> ========
>>
>> However it does not do what I anticipated. It indeed excludes everything but
>> the include part does not work - it only works for !/labels and !/scripts
>> directories (i.e. the first level directories). All other are still ignored -
>> so when I create file /db/data/load/base/bootstraponly/somefile.txt git still
>> ignores it...
>>
>> Any idea what I am doing wrong?
>
> The fourth bullet point of the "Pattern Format" section of the
> gitignore man page has this to say, which explains the behavior you're
> seeing:
>
>     An optional prefix "!" which negates the pattern; any matching
>     file excluded by a previous pattern will become included again.
>     It is not possible to re-include a file if a parent directory of
>     that file is excluded. Git doesn’t list excluded directories for
>     performance reasons, so any patterns on contained files have no
>     effect, no matter where they are defined.

This is true. To elaborate, if we have to recurse in excluded directories so that we can include some back, then the reason for excluding is already defeated as we may need to traverse the entire directory structure. However in this particular case where we do know in advance that only certain directories may have "re-include" rules, e.g. "db", "reports" or "scripts", we could keep going for a while. I think I attempted to do this in the past and failed (don't remember exactly why). Maybe I'll try again some time in future.

Another option is, if the user is willing to accept performance degradation (in many small repos, it does not matter anyway), then we could keep digging in.

-- 
Duy
Eric Sunshine· Mar 24, 2015, 01:55 UTC · re: Duy Nguyen · lore

Re: Git ignore help

On Mon, Mar 23, 2015 at 9:00 PM, Duy Nguyen <pclouds@gmail.com> wrote:
Show 23 quoted lines
> On Sun, Mar 22, 2015 at 10:33 AM, Eric Sunshine <sunshine@sunshineco.com> wrote:
>> On Fri, Mar 20, 2015 at 6:36 AM,  <mdconf@seznam.cz> wrote:
>>> Any idea what I am doing wrong?
>>
>> The fourth bullet point of the "Pattern Format" section of the
>> gitignore man page has this to say, which explains the behavior you're
>> seeing:
>>
>>     An optional prefix "!" which negates the pattern; any matching
>>     file excluded by a previous pattern will become included again.
>>     It is not possible to re-include a file if a parent directory of
>>     that file is excluded. Git doesn’t list excluded directories for
>>     performance reasons, so any patterns on contained files have no
>>     effect, no matter where they are defined.
>
> This is true. To elaborate, if we have to recurse in excluded
> directories so that we can include some back, then the reason for
> excluding is already defeated as we may need to traverse the entire
> directory structure. However in this particular case where we do know
> in advance that only certain directories may have "re-include" rules,
> e.g. "db", "reports" or "scripts", we could keep going for a while. I
> think I attempted to do this in the past and failed (don't remember
> exactly why). Maybe I'll try again some time in future.

I also was pretty sure that you had attempted this, but couldn't find a reference to it, so I didn't mention it in my response. However, with some more digging, I finally located it[1].

[1]: http://article.gmane.org/gmane.comp.version-control.git/259870
Show 5 quoted lines
> Another option is, if the user is willing to accept performance
> degradation (in many small repos, it does not matter anyway), then we
> could keep digging in.
> --
> Duy
Duy Nguyen· Mar 24, 2015, 09:39 UTC · re: Eric Sunshine · lore

Re: Git ignore help

On Tue, Mar 24, 2015 at 8:55 AM, Eric Sunshine <sunshine@sunshineco.com> wrote:
Show 9 quoted lines
>> e.g. "db", "reports" or "scripts", we could keep going for a while. I
>> think I attempted to do this in the past and failed (don't remember
>> exactly why). Maybe I'll try again some time in future.
>
> I also was pretty sure that you had attempted this, but couldn't find
> a reference to it, so I didn't mention it in my response. However,
> with some more digging, I finally located it[1].
>
> [1]: http://article.gmane.org/gmane.comp.version-control.git/259870

Thank you. I only looked at my repo and no branch name suggested it (if only there is google search for a git repository..). So I gave up because of performance reasons again but that was for enabling it unconditionaly. If we enable it via a config variable and the user is made aware of the performance implications, I guess it would be ok. So it's back in my back log.

-- 
Duy
mdconf@seznam.cz· Mar 24, 2015, 12:46 UTC · re: Duy Nguyen · lore

Re: Git ignore help

Hi Duy, Eric,
thx a lot. So net net - I can't really achieve this. It feels like something very basic and simple so pretty surprising but guess that's what it is :)
Normally, I'd expect that the functionality will behave like with similar other blacklist/whitelist functionalities in Linux - that the more specific definition overrides the less specific one. And when there are 2 on the same level then one (include or exclude) has the priority by default...
Anyways, what I am still not clear on is how come that including just the 'single level folder' work?
Duy, you wrote:
"This is true. To elaborate, if we have to recurse in excluded directories so that we can include some back, then the reason for excluding is already defeated as we may need to traverse the entire directory structure. However in this particular case where we do know in advance that only certain directories may have "re-include" rules, e.g. "db", "reports" or "scripts", we could keep going for a while."
... so according to that it sounds like including /db, /reports, /scripts should actually also NOT work. But it does work - i.e. when I add the following:

# exclude /*

# except !/db !/reports !/scripts

then any content within those 3 directories (and their sub directories) is included and not ignored...
It ONLY does not work when I add more levels - e.g.:
!/reports/something
In this case neither /reports nor /reports/something or any sub directory is included.
Martin
---------- Původní zpráva ----------
Od: Duy Nguyen 
Komu: Eric Sunshine 
Datum: 24. 3. 2015 10:40:33
Předmět: Re: Git ignore help
On Tue, Mar 24, 2015 at 8:55 AM, Eric Sunshine  wrote:
Show 9 quoted lines
>> e.g. "db", "reports" or "scripts", we could keep going for a while. I
>> think I attempted to do this in the past and failed (don't remember
>> exactly why). Maybe I'll try again some time in future.
>
> I also was pretty sure that you had attempted this, but couldn't find
> a reference to it, so I didn't mention it in my response. However,
> with some more digging, I finally located it[1].
>
> [1]: http://article.gmane.org/gmane.comp.version-control.git/259870

Thank you. I only looked at my repo and no branch name suggested it (if only there is google search for a git repository..). So I gave up because of performance reasons again but that was for enabling it unconditionaly. If we enable it via a config variable and the user is made aware of the performance implications, I guess it would be ok. So it's back in my back log.

-- 
Duy
Duy Nguyen· Mar 24, 2015, 13:11 UTC · re: mdconf@seznam.cz · lore

Re: Git ignore help

On Tue, Mar 24, 2015 at 7:46 PM,  <mdconf@seznam.cz> wrote:
Show 21 quoted lines
> Duy, you wrote:
>
> "This is true. To elaborate, if we have to recurse in excluded directories so that we can include some back, then the reason for excluding is already defeated as we may need to traverse the entire directory structure. However in this particular case where we do know in advance that only certain directories may have "re-include" rules, e.g. "db", "reports" or "scripts", we could keep going for a while."
>
> ... so according to that it sounds like including /db, /reports, /scripts should actually also NOT work. But it does work - i.e. when I add the following:
>
> # exclude
> /*
>
> # except
> !/db
> !/reports
> !/scripts
>
> then any content within those 3 directories (and their sub directories) is included and not ignored...
>
> It ONLY does not work when I add more levels - e.g.:
>
> !/reports/something
>
> In this case neither /reports nor /reports/something or any sub directory is included.

Yes. It's the subtlety of optimizing ;-) If you read the man page really carefully (*), "if the _parent_ directory of that file(**) is excluded" and the parent of these three directories is _not_ excluded.

(*) I'm not saying this is a good thing. Only docs such as language spec or RFCs need that level of attention. But I'm not a good document writer myself, can't blame others. Improvements are welcome though.

(**) "that file" should be "that file or directory" but I guess simplification here is ok

-- 
Duy
Eric Sunshine· Mar 24, 2015, 17:37 UTC · re: Duy Nguyen · lore

Re: Git ignore help

On Tue, Mar 24, 2015 at 5:39 AM, Duy Nguyen <pclouds@gmail.com> wrote:
Show 17 quoted lines
> On Tue, Mar 24, 2015 at 8:55 AM, Eric Sunshine <sunshine@sunshineco.com> wrote:
>>> e.g. "db", "reports" or "scripts", we could keep going for a while. I
>>> think I attempted to do this in the past and failed (don't remember
>>> exactly why). Maybe I'll try again some time in future.
>>
>> I also was pretty sure that you had attempted this, but couldn't find
>> a reference to it, so I didn't mention it in my response. However,
>> with some more digging, I finally located it[1].
>>
>> [1]: http://article.gmane.org/gmane.comp.version-control.git/259870
>
> Thank you. I only looked at my repo and no branch name suggested it
> (if only there is google search for a git repository..). So I gave up
> because of performance reasons again but that was for enabling it
> unconditionaly. If we enable it via a config variable and the user is
> made aware of the performance implications, I guess it would be ok. So
> it's back in my back log.

How much does a config variable actually help? In a sense, one could argue that this is already an opt-in feature since it requires crafting gitignore in a particular fashion. Existing projects which have (properly) functioning gitignore rules won't trigger this behavior. In many cases, Git already allows people to shoot themselves in the foot if they desire, thus, as long as the potential performance impact is properly documented, this could be considered another such instance.

Junio C Hamano· Mar 24, 2015, 20:54 UTC · re: Eric Sunshine · lore

Re: Git ignore help

Eric Sunshine <sunshine@sunshineco.com> writes:
Show 27 quoted lines
> On Tue, Mar 24, 2015 at 5:39 AM, Duy Nguyen <pclouds@gmail.com> wrote:
>> On Tue, Mar 24, 2015 at 8:55 AM, Eric Sunshine <sunshine@sunshineco.com> wrote:
>>>> e.g. "db", "reports" or "scripts", we could keep going for a while. I
>>>> think I attempted to do this in the past and failed (don't remember
>>>> exactly why). Maybe I'll try again some time in future.
>>>
>>> I also was pretty sure that you had attempted this, but couldn't find
>>> a reference to it, so I didn't mention it in my response. However,
>>> with some more digging, I finally located it[1].
>>>
>>> [1]: http://article.gmane.org/gmane.comp.version-control.git/259870
>>
>> Thank you. I only looked at my repo and no branch name suggested it
>> (if only there is google search for a git repository..). So I gave up
>> because of performance reasons again but that was for enabling it
>> unconditionaly. If we enable it via a config variable and the user is
>> made aware of the performance implications, I guess it would be ok. So
>> it's back in my back log.
>
> How much does a config variable actually help? In a sense, one could
> argue that this is already an opt-in feature since it requires
> crafting gitignore in a particular fashion. Existing projects which
> have (properly) functioning gitignore rules won't trigger this
> behavior. In many cases, Git already allows people to shoot themselves
> in the foot if they desire, thus, as long as the potential performance
> impact is properly documented, this could be considered another such
> instance.

Yeah, as I re-read that old thread, I really do not think anything wrong with the reasoning expressed in the proposed log message. It is pointless to hunt for "!do-not-exclude-me-please" in D/.gitignore when "D/" appears in .gitignore in the higher level, but if these two i.e.

	D/
        !D/do-not-exclude-me-please

appear together in .gitignore in the higher level, we can pay attention to that and pick up that single path. And doing so would be a lot more intuitive to the end user.

My comment in the thread was only about the documentation being unclear and not about the feature, but somehow we failed to follow-up the topic to completion X-<.

← back to recent threads