git-grep ignores untracked files

Currently, git-grep ignores untracked files in the working directory. For example:

    $ pattern=qazxswedc
    $ echo $pattern > foo
    $ git grep $pattern || echo not found
    not found

This is annoying and counter-intuitive, since it *does* search modified files with unstaged changes. More importantly, this is undocumented.

The situation is much worse when untracked files are given on the command line. For example, the following command warns me that a file does not exist, as expected.

    $ git grep $pattern does_not_exist
    fatal: ambiguous argument 'does_not_exist': unknown revision or path not in the working tree.
    Use '--' to separate paths from revisions

However, if I give it an untracked file, it just tells me there's no match.

    $ git grep $pattern foo || echo not found
    not found

In this case, I would expect the program to either search that file (would be nice!) or die with an error message (I'll take this for now). Furthermore, when '--' is given, even the non-existent case fails silently.

    $ git grep $pattern -- does_not_exist || echo not found
    not found

So, what I would I like to see? Ideally, something like the following series of patches:

1. Document the current behavior in the man page.

2. Issue a warning if any paths (pattern or non-pattern) fail to match
any files.  Ideally, this would work in all cases (work tree, --cached,
and trees).  If an untracked file matches a pattern that also matches
a tracked file, there would be no warning - not sure what's best in this
case.  This warning should probably be disable-able with a flag, say
--no-warn-if-unmatched.

3. If neither --cached nor trees are given, and a non-pattern filename
is given that does not not exist in the cache, search this file as well.
(That is, instead of issuing a warning, just search it.)  If the
filename is a directory, issue a warning.

4. Add an --untracked flag that causes grep to search *all* files in the
working directory, not just tracked ones.  (This includes searches with
and without a pathspec.)  The option would be incompatible with --cached
or given trees.

I post this wish list for several reasons. First, to see if anyone agrees with me. Second, to see if this patch series has a chance of ever being accepted. Third, to implant this idea into someones head so they write it for me :). I may try to do it myself, but it will take me a while.

To get started, here's a shot at #1.

Mark

---- 8< ----
From 697d97ca45b5c7abe8c84c3caae28cf839c668ac Mon Sep 17 00:00:00 2001
From: Mark Lodato <lodatom@gmail.com>
Date: Sun, 14 Feb 2010 19:28:55 -0500
Subject: [PATCH] Documentation: clarify untracked files and <path>

Note that only tracked files are searched, that non-matching paths are silently ignored, and that <path> is a glob(7) pattern.

Signed-off-by: Mark Lodato <lodatom@gmail.com>
---
 Documentation/git-grep.txt |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)

Strictly speaking, because git commands are about tracked files unless explicitly stated otherwise, the phrase "files in the work tree" already means "tracked" ones. It is understandable (but not excusable) that people who wrote manual assumed the readers would have learned that from the user manual without repeating. Adding explicit description would be a good thing to do.

I think rewording the paragraph to "... patterns in the tracked files in the work tree, blobs registered in the index file, ... or given tree objects." would be a good balance to strike. It gives enough information without making it too verbose.

I doubt we want to have "Only tracked files blah blah". Like all the normal git commands, "grep" is about tracked contents, and I don't think it would help to repeat the obvious like pathspec filter will act as a filter. "add <pathspec>" is an exception in that it _is_ about untracked paths and that is why you get warnings for unmatched ones.

    Side note: there will be --no-index option to let you run "git grep"
    over files in a random directory.

Please do *not* "see glob(7) for the format". Pathspec used for "grep" (and "ls-files") are "leading path match or glob(7)". E.g. "git grep frotz t/" looks for frotz in all files under "t/" recursively, and that does not have much to do with glob(7). If we do not have a description already, we may want to add these basics to git(1) or the user manual.

Thanks.

On Sun, Feb 14, 2010 at 8:45 PM, Junio C Hamano <gitster@pobox.com> wrote:

I think adding this information to the description of <path> would be sufficient. I'll send a new patch in a minute.

Ah, didn't see this. So, it looks like most of my requests were already done. However, it may still be a good idea to DWIM when none of --cached, --no-index, or trees are given but a pathspec is given that matches an untracked file in the working directory. For example:

$ git grep $pattern -- untracked_file

It is obvious that the user meant to specify --no-index. However, I'm not sure where to draw the line. What if they give tracked and untracked files, or if they given a glob pattern that matches both?

Still, the simple, "one path is given, it's not a pattern, and it matches exactly to an untracked file" case should probably run with --no-index.

Ah. This is a major inconsistency in the documentation. I have a lot to say about this, so I'll make this a separate thread. For the patch, I'll just reference git-add(1), which is the does have a description, and then we can fix this properly in a future patch.

Clarify that git-grep(1) searches only tracked files, and that each <path> is a glob, as in git-add(1). Add an example to show a simple use case for searching all .c and .h files.

The meta-variable <path> should be changed to an official term for a path glob, and the description for this should be in git(1), not git-add(1). However, we don't yet have such an official term, so just use <path> and reference git-add(1) for now.

Signed-off-by: Mark Lodato <lodatom@gmail.com>
---
 Documentation/git-grep.txt |   13 ++++++++++---
 1 files changed, 10 insertions(+), 3 deletions(-)

Mark Lodato <lodatom@gmail.com> writes:

Looks much better, even though I personally prefer a shorter "work tree" and also think the official name of the stuff is "pathspec".

Will queue until an alternative materializes (if ever) ;-).

Thanks.