Meta-variable naming convention in documentation

Currently, git's documentation and usage statements are inconsistent in the meta-variables (e.g., <path>) used to described positional arguments that denote a path. This post is an attempt to document the existing issues and to propose a potential solution.

To begin, I ran each of the git sub-commands in git 1.7.0 and documented its behavior. Each row contains the program, the meta-variables, and the type. If the man page and the '-h' usage statement differ, I list both. For example, git-annotate uses "file" in both the man page and the usage statement, while git-blame uses "<file>" in the man page but "file" in the usage statement.

The following accept exact filenames. No directory recursion or globing is applied. The first group die if the file does not exist, while the second group "filters" (f) -- they silently ignore non-matching files.

	program			man[/usage]		type
	-------			-----------		----
	git-annotate		file			file
	git-blame		<file>/file		file
	git-checkout-index	<file>			file
	git-hash-object		<file>			file
	git-mailsplit		<mbox>|<Maildir>	file
	git-merge-file		<current-file> ...	file
	git-merge-index		<file>/<filename>	file
	git-merge-one-file	<path>			file
	git-mergetool		<file>/file to merge	file
	git-mv			<args>/<source>		file
	git-pack-redundant	.pack filename/<...>	file
	git-send-email		<file>			file
	git-update-index	<file>			file
	git-am			<dir>			dir
	git-clone		<directory>		dir
	git-cvsserver		<directory>		dir
	git-daemon		<directory>		dir
	git-fetch-pack		<directory>		dir
	git-filter-branch	<directory>		dir
	git-format-patch	<dir>			dir
	git-fsck		<dir>			dir
	git-mailsplit		<directory>		dir
	git-peek-remote		<directory>		dir
	git-quiltimport		<dir>			dir
	git-receive-pack	<directory>		dir
	git-relink		<dir>			dir
	git-send-pack		<directory>		dir
	git-submodule		<path>			dir
	git-upload-archive	<directory>		dir
	git-upload-pack		<directory>		dir
	git-bundle		<file>			outfile
	git-mailinfo		<patch>			outfile
	git-pack-objects	base-name		outfile

	git-diff-tree		<path>			file	(f)
	git-ls-tree		paths/path		file	(f)

The following accept exact filenames or directories. If a directory, this matches all files within that directory recursively.

	git-archive		path			fdir

	git-bisect		<paths>/<pathspec>	fdir	(f)
	git-diff		<path>			fdir	(f)
	git-diff-files		<path>			fdir	(f)
	git-diff-index		<path>			fdir	(f)
	git-difftool		<path>/(nothing)	fdir	(f)
	git-log			<path>			fdir	(f)
	git-reset		<paths>			fdir	(f)
	git-rev-list		<paths>/paths		fdir	(f)
	git-show		(undoc)/<path>		fdir	(f)
	git-whatchanged		(undocumented)		fdir	(f)
	gitk			<path>			fdir	(f)

The following accept recursive directory matches or path globs.

	git-add			<filepattern>		glob
	git-checkout		<paths>/<file>		glob
	git-commit		<file>/<filepattern>	glob
	git-rm			<file>			glob

	git-clean		<path>/<paths>		glob	(f)
	git-grep		<path>/path		glob	(f)
	git-ls-files		<file>			glob	(f)
	git-status	<pathspec>/<filepattern>	glob?	(f)

* git-rm has --ignore-unmatch, which causes it to "filter"
* git-status only accepts globs for untracked files (currently a bug?)

Here are some examples showing what I mean by "type". (May be useful to have something like this in the documentation.)

    pattern            file  fdir  glob
    -------            ----  ----  ----
    path/to/base.ext   yes   yes   yes
    path                -    yes   yes
    *xt                 -     -    yes
    p*t                 -     -    yes
    base.ext            -     -     -
    b*                  -     -     -
    to                  -     -     -

* 'file' or 'dir'
* 'fdir' (f)
* 'glob'
* 'glob' (f)

(The only exceptions are git-diff-tree, git-ls-tree, and git-archive.)

Therefore, I suggest that we stick to a consistent naming convention for
these four groups, and document them in git(1).  Here's is a proposal,
but I'm not tied to these names.
* <file> for 'file'
* <dir> for 'dir'
* <path> for 'fdir' (f)
* <filepattern> for 'glob'
* <pathspec> for 'glob' (f)

We would have to come up with something for the three exceptions above, and some commands would benefit from more detailed meta-vars, such as "<source>... <destination>" for git-mv.

Additionally, it would be nice if all non-filtering commands had --ignore-unmatch, and all filtering commands had an option to die/warn if any arguments did not match. But, this is much more work than a simple documentation change.

Anyway, what do you think of this proposal? I am not sure that I have my partitioning correct, but I would like to see some sort of consistency in the documentation. I would be happy to implement whatever is decided.

Mark Lodato <lodatom@gmail.com> writes:

What does the gap between these two mean? Do you mean "bisect and later are not in the 'exact filenames or directories' group"?

In general, unless the command takes only one filesystem entity (e.g. in "format-patch -o <dir>", <dir> cannot be anything but a single directory; in "blame <file>", <file> cannot be anything but a single file), you never give a single "filename" to git command. Even when you say "git add Makefile", you are _not_ giving a filename that is "M' "a" "k" "e" ...; you are giving a _pattern_ to be matched with files git would find by traversing the filesystem. In the case of "Makefile", it may happen to match only one single file.

This pattern is called "pathspec", and commands that can take one pathspec can always take more than one.

Unfortunately, for historical reasons, there are two semantics of pathspec, and at least three implementations of pathspec logic.

 - diff family (diff, log, show, rev-list) does not support glob.  The
   pattern is matched either as a leading directory path, or exact name.

 - ls-files family (I think "clean" also uses the logic internally) does
   support glob.  The pattern is matched either as a leading directory
   path, exact name, or a glob.

 - grep implements the same logic as ls-files but uses a newer
   implementation better suited for optimized tree traversal.

And there are higher level commands that internally use logic from either diff family or ls-files family. You can guess which pathspec is used if you think about how you would implement what they do. For example:

 - "add <pathspec>" traverses the work tree using ls-files logic and adds
   found files to the index.

 - "add -u <pathspec>" compares the index and the work tree using
   diff-files logic and adds paths with differences to the index.

 - "status" uses "diff-index --cached" logic to come up with 'Changed to
   be committed' list, "diff-files" logic to come up with 'Changed but not
   updated' list, and "ls-files" logic to list 'Untracked' files.

Unifying the two different semantics of pathspecs is one of the suggested topics for GSoC, by the way.

I think it would make sense to document which ones are concrete paths (e.g. "blame takes a filename" vs "diff takes zero or more pathspecs"), but it would not make much sense to document the two different pathspecs. The effort is better spent at fixing the difference --- obviously we would eventually want to be able to say "git diff 'lib/*.h'".

On Fri, Feb 26, 2010 at 12:42 AM, Junio C Hamano <gitster@pobox.com> wrote:

Each of the three types is divided in two, separated by a blank line. The first group, all without (f) at the end, die if one of the paths given on the command line does not exist. The second group, all with (f) at the end, do not complain if any of the paths do not match anything. I called the latter group "filtering" for lack of a better term.

So, git-archive, git-add, git-checkout, git-commit, git-rm all take pathspecs (the first using diff logic, the rest using ls-files logic), but they complain if any of the pathspecs do not match. It's probably not worth documenting this, but it may be worth implementing --ignore-unmatch for all but git-rm (which already has it.)

Similarly, it may be a nice option to have a --warn-unmatch option (or a configuration variable) to warn (or possibly die) if any of the pathspecs hit nothing. Then again, maybe it's not worth the trouble.

It appears that git-diff-tree and git-ls-tree take some sort of pathspec - it ignores unmatching patterns, but it does not allow globbing or directory matches.

In this case, I suggest that we use the same meta-variable always: <file> or <dir>, or a more specific thing like <source>.

In these cases, I think we should always use the meta-variable <pathspec> - never <path>, <file>, etc.

You're right, it's not worth having two different meta-variables, but I do think it is worth noting in the documentation. In git(1), we could have something like the following:

<pathspec>::
	Indicates a pattern for filtering paths.  Matches are either exact,
	a leading directory, or a glob(7) pattern on the entire path.
	For example, "doc/help.txt" matches "doc/help.txt", "doc",
	"*.txt", and "d*t", but not "do", "help.txt", or "help.*".

<dir>::
<file>::
	Indicates a physical file or directory relative to
	the current directory.

(Note: the current document says "almost always relative to the root of the tree structure `GIT_INDEX_FILE` describes." Is this true?)

Then, for the diff logic commands, we could document that they do not accept globs. For example, in git-diff(1):

<pathspec>...::
	If given, limit the diff to paths matching the given parameters.
	(Does not accept glob(7) patterns.)

Once the logic is unified, and these commands do accept globs, we can just remove the note.

Mark

On a similar note, what do you think about dropping the term <tree-ish> and just using <tree> everywhere? The only command that only accepts a <tree>, and not a <tree-ish>, is git-commit-tree. For example, `git commit-tree master' fails, but `git commit-tree master^{tree}' works. This can easily be written in the documentation, or the program can be fixed so it also accepts a <tree-ish> like everything else. (git-grep(1) uses the term <tree>, but this should be <tree-ish>.)

I imagine a similar thing can be done with <commit> vs <commit-ish> vs <committish>, but I haven't verified this. <commit-ish> is only used in git(1) and builtin-revert.c, and <committish> is only used in git-describe(1), git-fast-import(1), git-name-rev(1), git-shortlog(1), gitcli(1), and builtin-describe.c. I would guess that all the other commands that say <commit> really accept a <commit-ish>, but perhaps this is not true.

I also think <rev> should be replaced with <commit>, unless this means something different.

If you give the go-ahead, I'll work on a patch to do this.

Cheers, Mark