--relative option to say which subdirectory to pretend to be in, i.e. adds the --relative=<path> version) it looks like this option was intended for patch (-p) output format.
There was added support also for raw output format, so both 'git diff-tree' and 'git diff --raw' works with --relative option, but support for this is buggy, and in my opinion wrong way around.
The '--relative[=<path>]' option works currently like this. First, if git command is invoked in subdirectory the diffopts structure gets set prefix and prefix_length (in init_revisions). If --relative[=<path>] option is passed, git sets RELATIVE_NAME flag, and if there is argument, prefix is set to it. Later git removes prefix (sets it to NULL) if RELATIVE_NAME option is not set.
What's important in this step is that prefix is set without any normalization from --relative=<path> argument, while (from what I understand) if it is set from current directory i.e. with --relative option without argument, it is set with trailing slash. So using --relative=sub instead of --relative=sub/ might be thought as user error... but I think here it is lack of robustness in the API.
Then comes the filtering part. In functions such as diff_change or stuff_change, if path(s) does not begin with prefix, they are simply skipped. This solution limits --relative[=<path>] to work only with *recursive* (full tree) output, such as patch output format, or "git diff", or "git diff-tree -r" (and "git diff-tree -t", which implies "-r").
Last there is filename munging, done using strip_prefix function.
This is done using prefix_length only, and that is the cause of
the bug:
$ git diff-tree --abbrev -r --raw HEAD --relative=sub
a3a8425fe5496c61921010cb1e7b455a1f52bb86
:100644 100644 d90bda0... cefcae0... M /quux
if one uses '--relative=sub' instead of '--relative=sub/'.
What I'd like to see for the raw output format is to work with --relative[=<path>] to work as if <path> was top directory of repository. For example for diff between two trees
$ git diff-tree A B --relative=sub/
would be equivalent to running
$ git diff-tree A:sub/ B:sub/
*This* could be done, I think, by modifying diff_tree_sha1 to do a diff betweem A:sub/ and B:sub/ (taking 'sub/' from prefix) and unsetting prefix (setting prefix to NULL and prefix_length to 0). But that would work only in the case that can be reducted to diff between two tree objects. This wouldn't work for diff in raw output format between tree and working area, tree and index, or index and working area.
Is the idea of automagically translating <sha1> into <sha1>:<prefix> to support --relative / relative=<prefix> well in raw diff output format a good idea? Or should I search for another solution.
I also do not know code enough (and it is not simple) to guess how one would go with the same result for diff between trees, index, and working area files.
BTW. the approach proposed here has the advantage that for B:<sub>, if <sub> does not exist in B, we can try to do what 'subtree' merge strategy does (and what wholesame directory rename detection did), namely try to find given directory under different path (like for example subtree-merged git-gui and gitk).
Jakub Narebski