threads / patch / 26997

patchRe: [PATCH] Documentation: enhance gitignore whitelist example

Subject: Re: [PATCH] Documentation: enhance gitignore whitelist example

## tl;dr

9 messages between Apr 5, 2011 and Apr 5, 2011. Diffs are folded; open one to read it.

replies: 8people: 4as markdown or json

Eric Blake· Apr 5, 2011, 19:36 UTC · lore

[PATCH] Documentation: enhance gitignore whitelist example

I was trying to whitelist a single file pattern in a directory that I was otherwise content to ignore, but when I tried:

/m4/ !/m4/virt-*.m4

then 'git add' kept warning me that I had to use -f. I finally figured out that ignoring a directory is much different than ignoring all files in a directory, when it comes to later negation patterns:

/m4/* !/m4/virt-*.m4

Improving the documentation will help others learn from my mistake.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 Documentation/gitignore.txt |   19 +++++++++++++++++--
 1 files changed, 17 insertions(+), 2 deletions(-)
Show changes to Documentation/gitignore.txt +17 −2
diff --git a/Documentation/gitignore.txt b/Documentation/gitignore.txt
index 2e7328b..2f49989 100644
--- a/Documentation/gitignore.txt
+++ b/Documentation/gitignore.txt
@@ -70,7 +70,9 @@ PATTERN FORMAT
  - An optional prefix '!' which negates the pattern; any
    matching file excluded by a previous pattern will become
    included again.  If a negated pattern matches, this will
-   override lower precedence patterns sources.
+   override lower precedence patterns sources.  However, a
+   file negation does not override a path that has already
+   been excluded by a directory match.

  - If the pattern ends with a slash, it is removed for the
    purpose of the following description, but it would only find
@@ -87,7 +89,8 @@ PATTERN FORMAT

  - Otherwise, git treats the pattern as a shell glob suitable
    for consumption by fnmatch(3) with the FNM_PATHNAME flag:
-   wildcards in the pattern will not match a / in the pathname.
+   wildcards in the pattern will not match a / in the pathname,
+   and do not ignore files with a leading . in the pathname.
    For example, "Documentation/{asterisk}.html" matches
    "Documentation/git.html" but not "Documentation/ppc/ppc.html"
    or "tools/perf/Documentation/perf.html".
@@ -116,8 +119,11 @@ EXAMPLES
     [...]
     # Untracked files:
     [...]
+    #       Documentation/build
     #       Documentation/foo.html
     #       Documentation/gitignore.html
+    #       build/log
+    #       build/.file
     #       file.o
     #       lib.a
     #       src/internal.o
@@ -125,6 +131,10 @@ EXAMPLES
     $ cat .git/info/exclude
     # ignore objects and archives, anywhere in the tree.
     *.[oa]
+    # ignore files in the immediate child directory build,
+    /build/*
+    # except for the log.
+    !/build/log
     $ cat Documentation/.gitignore
     # ignore generated html files,
     *.html
@@ -134,10 +144,15 @@ EXAMPLES
     [...]
     # Untracked files:
     [...]
+    #       Documentation/build
     #       Documentation/foo.html
+    #       build/log
     [...]
 --------------------------------------------------------------

+Note that using `!/build/log' works with an earlier `/build/*' but
+would have no effect if there were an earlier `/build/'.
+
 Another example:

 --------------------------------------------------------------

-- 
1.7.4
Jonathan Nieder· Apr 5, 2011, 19:40 UTC · re: Eric Blake · lore
Eric Blake wrote:
Show 16 quoted lines
> I was trying to whitelist a single file pattern in a directory
> that I was otherwise content to ignore, but when I tried:
> 
> /m4/
> !/m4/virt-*.m4
> 
> then 'git add' kept warning me that I had to use -f.  I finally
> figured out that ignoring a directory is much different than ignoring
> all files in a directory, when it comes to later negation patterns:
> 
> /m4/*
> !/m4/virt-*.m4
> 
> Improving the documentation will help others learn from my mistake.
> 
> Signed-off-by: Eric Blake <eblake@redhat.com>
Yes.
Acked-by: Jonathan Nieder <jrnieder@gmail.com>

Cc-ing Hannes, in case he has thoughts on how to explain this more intuitively.

Show 71 quoted lines
> ---
>  Documentation/gitignore.txt |   19 +++++++++++++++++--
>  1 files changed, 17 insertions(+), 2 deletions(-)
> 
> diff --git a/Documentation/gitignore.txt b/Documentation/gitignore.txt
> index 2e7328b..2f49989 100644
> --- a/Documentation/gitignore.txt
> +++ b/Documentation/gitignore.txt
> @@ -70,7 +70,9 @@ PATTERN FORMAT
>   - An optional prefix '!' which negates the pattern; any
>     matching file excluded by a previous pattern will become
>     included again.  If a negated pattern matches, this will
> -   override lower precedence patterns sources.
> +   override lower precedence patterns sources.  However, a
> +   file negation does not override a path that has already
> +   been excluded by a directory match.
> 
>   - If the pattern ends with a slash, it is removed for the
>     purpose of the following description, but it would only find
> @@ -87,7 +89,8 @@ PATTERN FORMAT
> 
>   - Otherwise, git treats the pattern as a shell glob suitable
>     for consumption by fnmatch(3) with the FNM_PATHNAME flag:
> -   wildcards in the pattern will not match a / in the pathname.
> +   wildcards in the pattern will not match a / in the pathname,
> +   and do not ignore files with a leading . in the pathname.
>     For example, "Documentation/{asterisk}.html" matches
>     "Documentation/git.html" but not "Documentation/ppc/ppc.html"
>     or "tools/perf/Documentation/perf.html".
> @@ -116,8 +119,11 @@ EXAMPLES
>      [...]
>      # Untracked files:
>      [...]
> +    #       Documentation/build
>      #       Documentation/foo.html
>      #       Documentation/gitignore.html
> +    #       build/log
> +    #       build/.file
>      #       file.o
>      #       lib.a
>      #       src/internal.o
> @@ -125,6 +131,10 @@ EXAMPLES
>      $ cat .git/info/exclude
>      # ignore objects and archives, anywhere in the tree.
>      *.[oa]
> +    # ignore files in the immediate child directory build,
> +    /build/*
> +    # except for the log.
> +    !/build/log
>      $ cat Documentation/.gitignore
>      # ignore generated html files,
>      *.html
> @@ -134,10 +144,15 @@ EXAMPLES
>      [...]
>      # Untracked files:
>      [...]
> +    #       Documentation/build
>      #       Documentation/foo.html
> +    #       build/log
>      [...]
>  --------------------------------------------------------------
> 
> +Note that using `!/build/log' works with an earlier `/build/*' but
> +would have no effect if there were an earlier `/build/'.
> +
>  Another example:
> 
>  --------------------------------------------------------------
> -- 
> 1.7.4
>
Johannes Sixt· Apr 5, 2011, 21:15 UTC · re: Jonathan Nieder · lore
On Dienstag, 5. April 2011, Jonathan Nieder wrote:
Show 9 quoted lines
> Eric Blake wrote:
> > @@ -70,7 +70,9 @@ PATTERN FORMAT
> >   - An optional prefix '!' which negates the pattern; any
> >     matching file excluded by a previous pattern will become
> >     included again.  If a negated pattern matches, this will
> > -   override lower precedence patterns sources.
> > +   override lower precedence patterns sources.  However, a
> > +   file negation does not override a path that has already
> > +   been excluded by a directory match.

I don't think this is the right place to explain this caveat. Here we describe the format and behavior of the patterns in a rather formal manner.

Show 7 quoted lines
> > @@ -87,7 +89,8 @@ PATTERN FORMAT
> >
> >   - Otherwise, git treats the pattern as a shell glob suitable
> >     for consumption by fnmatch(3) with the FNM_PATHNAME flag:
> > -   wildcards in the pattern will not match a / in the pathname.
> > +   wildcards in the pattern will not match a / in the pathname,
> > +   and do not ignore files with a leading . in the pathname.
I don't think this is correct. * matches .gitignore. I tried it.
Show 20 quoted lines
> > @@ -116,8 +119,11 @@ EXAMPLES
> >      [...]
> >      # Untracked files:
> >      [...]
> > +    #       Documentation/build
> >      #       Documentation/foo.html
> >      #       Documentation/gitignore.html
> > +    #       build/log
> > +    #       build/.file
> >      #       file.o
> >      #       lib.a
> >      #       src/internal.o
> > @@ -125,6 +131,10 @@ EXAMPLES
> >      $ cat .git/info/exclude
> >      # ignore objects and archives, anywhere in the tree.
> >      *.[oa]
> > +    # ignore files in the immediate child directory build,
> > +    /build/*
> > +    # except for the log.
> > +    !/build/log
Doesn't this example give the false impression that you could do
	/foo/*
	!/foo/bar/baz
and have foo/bar/baz not ignored? But it is still ignored.
I propose a paragraph like this in the NOTES section:

--- 8< --- When a directory is ignored, it is not possible to un-ignore a single file somewhere in the directory using another pattern. E.g., with the patterns

-------------- /build/ !/build/tests/results --------------

the file "build/tests/results" is still ignored because when a directory is ignored, its contents are never investigated. In a situation where a few exceptions in an otherwise ignored hierarchy are needed, the recommended procedure is to specify to ignore the root of the hierarchy and then to 'git add -f' the exceptional files. Subsequent changes to the files will not be ignored. --- 8< ---

-- Hannes
Eric Blake· Apr 5, 2011, 21:23 UTC · re: Johannes Sixt · lore
On 04/05/2011 03:15 PM, Johannes Sixt wrote:
Show 9 quoted lines
>>> @@ -87,7 +89,8 @@ PATTERN FORMAT
>>>
>>>   - Otherwise, git treats the pattern as a shell glob suitable
>>>     for consumption by fnmatch(3) with the FNM_PATHNAME flag:
>>> -   wildcards in the pattern will not match a / in the pathname.
>>> +   wildcards in the pattern will not match a / in the pathname,
>>> +   and do not ignore files with a leading . in the pathname.
> 
> I don't think this is correct. * matches .gitignore. I tried it.

That was my point. * _does_ match .gitignore, even though for normal shell globs, FNM_PERIOD is set and * does not match .gitignore. That is, while in the shell 'dir/*' only matches non-dot files, in .gitignore it matches all files including dot-files.

Any ideas for a better way to word that?
Show 17 quoted lines
> I propose a paragraph like this in the NOTES section:
> 
> --- 8< ---
> When a directory is ignored, it is not possible to un-ignore a single file 
> somewhere in the directory using another pattern. E.g., with the patterns
> 
> --------------
> /build/
> !/build/tests/results
> --------------
> 
> the file "build/tests/results" is still ignored because when a directory is 
> ignored, its contents are never investigated. In a situation where a few 
> exceptions in an otherwise ignored hierarchy are needed, the recommended 
> procedure is to specify to ignore the root of the hierarchy and then to 'git 
> add -f' the exceptional files. Subsequent changes to the files will not be 
> ignored.

Yeah, but then you have to 'git add -f path/to/file' them every time you change them, or use the sledgehammer of 'git add .'.

Does it make any better sense to document:
  /build/*
  !/build/*/
  /build/*/*
  !/build/foo/baz

which ignores all files in build, then un-ignores directories, then ignores all files in subdirectories of build except for the desired multi-level file under build? At which point you no longer need 'git add -f', but can simply do 'git add build' to pick up /build/foo/baz in one go without warning?

-- 
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org
Jonathan Nieder· Apr 5, 2011, 21:41 UTC · re: Eric Blake · lore
Eric Blake wrote:
> Yeah, but then you have to 'git add -f path/to/file' them every time you
> change them
No, I don't believe that's true.
 $ git add -f git.o
 $ >git.o
 $ git add git.o

.gitignore only protects against starting to track a file that was previously untracked.

I do tend to use exceptions in .gitignore anyway, since (1) it allows, as a sanity check,

 $ git ls-files -i --exclude-standard
and (2) to import from or compare to a tarball, one can do
 $ git rm -rf .
 $ git add .
Eric Blake· Apr 5, 2011, 21:49 UTC · re: Jonathan Nieder · lore
On 04/05/2011 03:41 PM, Jonathan Nieder wrote:
Show 10 quoted lines
> Eric Blake wrote:
> 
>> Yeah, but then you have to 'git add -f path/to/file' them every time you
>> change them
> 
> No, I don't believe that's true.
> 
>  $ git add -f git.o
>  $ >git.o
>  $ git add git.o
Aha - it's that pesky dir/ vs. dir/* biting me, yet again:
$ mkdir -p /tmp/blah
$ cd /tmp/blah
$ git init
Initialized empty Git repository in /tmp/blah/.git/
$ mkdir sub
$ > sub/file
$ git add sub/file
$ git commit -a -m 'one'
[master (root-commit) 645ee5a] one
 0 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 sub/file
$ printf 'sub/*\n!sub/file\n' > .gitignore
$ touch sub/file2
$ echo hi > sub/file
$ git status
# On branch master
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working
directory)
#
#	modified:   sub/file
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#	.gitignore
no changes added to commit (use "git add" and/or "git commit -a")
$ git add sub
$ git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#	modified:   sub/file
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#	.gitignore
$ git reset
Unstaged changes after reset:
M	sub/file
$ printf 'sub/\n!sub/file\n' > .gitignore
$ git add sub
The following paths are ignored by one of your .gitignore files:
sub
Use -f if you really want to add them.
fatal: no files added
$ git add sub/file
The following paths are ignored by one of your .gitignore files:
sub
Use -f if you really want to add them.
fatal: no files added
$ git status
# On branch master
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working
directory)
#
#	modified:   sub/file
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#	.gitignore
no changes added to commit (use "git add" and/or "git commit -a")
$ git add .
$ git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#	new file:   .gitignore
#	modified:   sub/file
#
> 
> .gitignore only protects against starting to track a file that was
> previously untracked.

Not quite. When filtering a directory, it also protects against changes to tracked files in that directory. And that is what has been throwing me off, which is why we need a doc change (or possibly even a behavior change).

-- 
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org
Junio C Hamano· Apr 5, 2011, 21:51 UTC · re: Eric Blake · lore
Eric Blake <eblake@redhat.com> writes:
Show 17 quoted lines
> On 04/05/2011 03:15 PM, Johannes Sixt wrote:
>>>> @@ -87,7 +89,8 @@ PATTERN FORMAT
>>>>
>>>>   - Otherwise, git treats the pattern as a shell glob suitable
>>>>     for consumption by fnmatch(3) with the FNM_PATHNAME flag:
>>>> -   wildcards in the pattern will not match a / in the pathname.
>>>> +   wildcards in the pattern will not match a / in the pathname,
>>>> +   and do not ignore files with a leading . in the pathname.
>> 
>> I don't think this is correct. * matches .gitignore. I tried it.
>
> That was my point.  * _does_ match .gitignore, even though for normal
> shell globs, FNM_PERIOD is set and * does not match .gitignore.  That
> is, while in the shell 'dir/*' only matches non-dot files, in .gitignore
> it matches all files including dot-files.
>
> Any ideas for a better way to word that?

Instead of "and do not ignore files with a leading", say "but will match a dot '.'". You are talking about the rule for wildcards to match or not to match the pathname and there is no room for the word "ignore" to come into play in this sentence.

Junio C Hamano· Apr 5, 2011, 21:39 UTC · re: Johannes Sixt · lore
Johannes Sixt <j6t@kdbg.org> writes:
Show 9 quoted lines
>> > @@ -87,7 +89,8 @@ PATTERN FORMAT
>> >
>> >   - Otherwise, git treats the pattern as a shell glob suitable
>> >     for consumption by fnmatch(3) with the FNM_PATHNAME flag:
>> > -   wildcards in the pattern will not match a / in the pathname.
>> > +   wildcards in the pattern will not match a / in the pathname,
>> > +   and do not ignore files with a leading . in the pathname.
>
> I don't think this is correct. * matches .gitignore. I tried it.
It is badly phrased to begin with.

It shouldn't say "do not ignore", which is determined by the presense or the lack of the leading '!' before the pattern. The pattern either matches or does not match paths that begin with dot.

I think we pass FNM_PATHNAME but not FNM_PERIOD, so * would match a period hence ".gitignore". What Eric wrote may be correct in the sense that "the pattern matcher does not _ignore_ .gitignore as not matching but catches it with '*'", but as the result the path ends up getting ignored ;-)

> I propose a paragraph like this in the NOTES section:

I think it makes sense to have something like that, but please state it like "to do Y, do X and Z" (optionally followed by "with just X, Y won't happen because..., hence you need to do Z as well"), instead of starting with "you cannot do Y with only X".

IOW, start from a positive and helpful recipe, and then explain why "you cannot do Y with only X" after that.

Show 16 quoted lines
> --- 8< ---
> When a directory is ignored, it is not possible to un-ignore a single file 
> somewhere in the directory using another pattern. E.g., with the patterns
>
> --------------
> /build/
> !/build/tests/results
> --------------
>
> the file "build/tests/results" is still ignored because when a directory is 
> ignored, its contents are never investigated. In a situation where a few 
> exceptions in an otherwise ignored hierarchy are needed, the recommended 
> procedure is to specify to ignore the root of the hierarchy and then to 'git 
> add -f' the exceptional files. Subsequent changes to the files will not be 
> ignored.
> --- 8< ---
Junio C Hamano· Apr 5, 2011, 20:56 UTC · re: Eric Blake · lore
Eric Blake <eblake@redhat.com> writes:
Show 5 quoted lines
> I was trying to whitelist a single file pattern in a directory
> that I was otherwise content to ignore, but when I tried:
>
> /m4/
> !/m4/virt-*.m4

Please always indent displayed examples in commit log messages for readability.

Show 26 quoted lines
> then 'git add' kept warning me that I had to use -f.  I finally
> figured out that ignoring a directory is much different than ignoring
> all files in a directory, when it comes to later negation patterns:
>
> /m4/*
> !/m4/virt-*.m4
>
> Improving the documentation will help others learn from my mistake.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
>  Documentation/gitignore.txt |   19 +++++++++++++++++--
>  1 files changed, 17 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/gitignore.txt b/Documentation/gitignore.txt
> index 2e7328b..2f49989 100644
> --- a/Documentation/gitignore.txt
> +++ b/Documentation/gitignore.txt
> @@ -70,7 +70,9 @@ PATTERN FORMAT
>   - An optional prefix '!' which negates the pattern; any
>     matching file excluded by a previous pattern will become
>     included again.  If a negated pattern matches, this will
> -   override lower precedence patterns sources.
> +   override lower precedence patterns sources.  However, a
> +   file negation does not override a path that has already
> +   been excluded by a directory match.

It may be better to say "However X doesn't do Y" than not saying anything, but can't we phrase this more like "If you want to do Y, you need to do Z also/instead"? It would be much more useful for people who are looking for a way to do Y if you didn't stop at saying "X is not the way to do it", and said "Do X and Z if you want to achieve Y", no?

On the other hand, if you are trying to explain why X doesn't do Y, the above would need a bit more explanation (e.g. "when a directory matches an ignore pattern, it tells git not to descend into the directory to find ignored or unignored paths in it" or something like that).

Show 8 quoted lines
> @@ -125,6 +131,10 @@ EXAMPLES
>      $ cat .git/info/exclude
>      # ignore objects and archives, anywhere in the tree.
>      *.[oa]
> +    # ignore files in the immediate child directory build,
> +    /build/*
> +    # except for the log.
> +    !/build/log

In the patch form it is clear these two lines go together, but the correspondence does not stand out in the text after the patch is applied.

Perhaps doing it like this would make it clearer?
> +    # ignore files in the immediate child directory build, ...
> +    /build/*
> +    # ... except for the log.
> +    !/build/log

← back to recent threads