git diff across submodules

Hello,
  Is there any nice way to get a diff and/or diffstat of both a project and its submodules between two revisions of the main project?

Something like 'git diff --stat tag_a tag_b' but also including the diffstat on the submodule from the revision tied to in tag_a to the revision tied to tag_b.  A few shell backflips will do it, but this seems like I'm missing something.

thanks, -Kenny

Am 18.09.2012 05:12, schrieb Kenny Simpson:

Unfortunately that isn't possible yet, "git diff" still has to learn the --recurse-submodules option. And as far as I know, nobody is currently working on that.

Jens Lehmann <Jens.Lehmann@web.de> writes:

I do not think it is _that_ unfortunate, at least for two reasons.

When I made something a submodule, not a plain vanilla directory, I did want it to be treated differently from a collection of files in random states (which is what a directory is), but as a logical unit. If I bind another project 'frotz' at my path 'lib/frotz' and ask for diff of two versions of my superproject, e.g.

    $ git diff v1.0 v1.1

I am more interested in finding out that we used to use v2.5 of 'frotz' back when we were at v1.0 but in our v1.1 we downgraded it to v2.4, perhaps because we found a regression in the 'frotz' library, rather than the whole reverse differences between v2.4 and v2.5 of the frotz project. That difference, when I want to, I can get by going to that submodule and grab it myself with

    $ cd lib/frotz &7 git diff v2.4 v2.5

I also suspect that you do not have to change "git diff" at all to show the patch recursively by using the attribute mechanism (look in Documentation/gitattributes.text for a string GIT_EXTERNAL_DIFF). It might be just as simple as doing this:

	echo >.gitattributes "/lib/frotz diff=subrecurse" 
	git config diff.subrecurse.command $HOME/bin/diff-subrecurse
        cat >$HOME/bin/diff-subrecurse <<\-EOF
	#!/bin/sh
        path=$1 old_hex=$3 new_hex=$6
        unset GIT_DIR
        cd "$path" || exit 1
        git diff "$old_hex" "$new_hex"        
        EOF
        chmod +x $HOME/bin/diff-subrecurse

The corner cases like "new submodule", "removed submodule" are left as an exercise to the reader ;-)

Junio C Hamano <gitster@pobox.com> writes:

It turns out that essentially the above outline I concocted in my MUA is usable almost as-is.

Here is what I ended up with.

 * In .git/config of the superproject, I added this:

        [diff "submodule-recurse"]
		command = src/bin/diff-submodule-recurse

 * In the superproject, src/bin/diff-submodule-recurse has this
   (this is probably whitespace damaged---the lines must be indented
   by HT for the here document to correctly work):

        #!/bin/sh
        # $1   $2       $3      $4       $5       $6      $7
        # path old-file old-hex old-mode new-file new-hex new-mode

        case "$#,$4,$7" in
        7,160000,160000) ;;
        *)      echo "diff --git a/$1 b/$1"
                echo "(punt)"
                exit
                ;;
        esac
        unset GIT_DIR
        cd "$1" || {
                cat <<-\EOF
                diff --git a/$1 b/$1
                (cannot chdir to $1)
		-Subproject commit $3
		+Subproject commit $6
                EOF
        }
        git --no-pager diff --src-prefix="s/$1/" --dst-prefix="m/$1/" "$3" "$6"

 * In .gitattributes of the superproject, I have this:

        /var	diff=submodule-recurse

The superproject in this case is a repository to control what I have in my $HOME directory (e.g. it has src/dot/Makefile that builds and installs the appropriate dotfiles, src/bin/Makefile that builds and installs to $HOME/bin, etc.), and one subdirectory, 'var', is a submodule that is only cloned to some but not all machines I clone this superproject to.

With this setting, things like

	$ git diff HEAD~20

show differences with recursion into the var/ submodule just fine.

Am 20.09.2012 00:31, schrieb Junio C Hamano:

That's pretty cool! Even though diff options like --stat and --name-only still won't take into account what happened inside the submodule this approach makes it possible to see the diff recursively. Wouldn't it make sense add this script to contrib (after teaching it new and removed submodules and documenting its use in a few lines after the shebang)?

Jens Lehmann <Jens.Lehmann@web.de> writes:

A few things somebody may want to work on while doing that "few lines of documentation" I know about are:

 * From the core side, pass options that are releavant when
   generating patch (i.e. with p) in environment variables to the
   external diff script;

 * Not using "s/$1" and "m/$1" as prefix; instead, pass src/dst
   prefix values (i.e. s/ and m/) from the core side in environment
   variables, and make the external diff script itself aware of
   possibly nested submodules, e.g.

    SUBMODULE_PATH="${SUBMODULE_PATH}$1"
    export SUBMODULE_PATH
    exec git --no-pager diff -p \
    	--src-prefix="$SRC_PREFIX/$SUBMODULE_PATH" \
    	--dst-prefix="$DST_PREFIX/$SUBMODULE_PATH" "$3" "$6"

After people gain sufficient experience with it, as the next step, we can think about how to handle --stat and other options when we are run without -p (currently the attribute mechanism would not trigger) and then we can call the result a native "diff that recurses into submodules" that people can use without setting up the attributes based mechanism.