{"thread":{"id":"31580","subject":"git diff across submodules","startedAt":"2012-09-18T03:12:39Z","lastAt":"2012-09-20T20:52:41Z","messageCount":6,"participants":["Kenny Simpson","Jens Lehmann","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"199324","messageId":"1347937959.96154.YahooMailClassic@web39403.mail.mud.yahoo.com","threadId":"31580","inReplyTo":null,"subject":"git diff across submodules","fromName":"Kenny Simpson","fromEmail":"theonetruekenny@yahoo.com","sentAt":"2012-09-18T03:12:39Z","receivedAt":"2012-09-18T03:12:39Z","isPatch":false,"sender":{"key":"theonetruekenny@yahoo.com","avatar":null},"body":"Hello,\n Is there any nice way to get a diff and/or diffstat of both a project and its submodules between two revisions of the main project?\n\nSomething like 'git diff --stat tag_a tag_b' but also including the diffstat on the submodule from the revision tied to in tag_a to the revision tied to tag_b. A few shell backflips will do it, but this seems like I'm missing something.\n\nthanks,\n-Kenny\n"},{"id":"199386","messageId":"5058C53B.5040401@web.de","threadId":"31580","inReplyTo":"1347937959.96154.YahooMailClassic@web39403.mail.mud.yahoo.com","subject":"Re: git diff across submodules","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2012-09-18T19:02:19Z","receivedAt":"2012-09-18T19:02:19Z","isPatch":false,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 18.09.2012 05:12, schrieb Kenny Simpson:\n> Is there any nice way to get a diff and/or diffstat of both a project and its submodules between two revisions of the main project?\n> \n> Something like 'git diff --stat tag_a tag_b' but also including the diffstat on the submodule from the revision tied to in tag_a to the revision tied to tag_b. A few shell backflips will do it, but this seems like I'm missing something.\n\nUnfortunately that isn't possible yet, \"git diff\" still has to learn the\n--recurse-submodules option. And as far as I know, nobody is currently\nworking on that.\n"},{"id":"199391","messageId":"7vobl3m8pb.fsf@alter.siamese.dyndns.org","threadId":"31580","inReplyTo":"5058C53B.5040401@web.de","subject":"Re: git diff across submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-09-18T19:28:48Z","receivedAt":"2012-09-18T19:28:48Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jens Lehmann writes:\n\n> Am 18.09.2012 05:12, schrieb Kenny Simpson:\n>> Is there any nice way to get a diff and/or diffstat of both a project and its submodules between two revisions of the main project?\n>> \n>> Something like 'git diff --stat tag_a tag_b' but also including the diffstat on the submodule from the revision tied to in tag_a to the revision tied to tag_b. A few shell backflips will do it, but this seems like I'm missing something.\n>\n> Unfortunately that isn't possible yet, \"git diff\" still has to learn the\n> --recurse-submodules option. And as far as I know, nobody is currently\n> working on that.\n\nI do not think it is _that_ unfortunate, at least for two reasons.\n\nWhen I made something a submodule, not a plain vanilla directory, I\ndid want it to be treated differently from a collection of files in\nrandom states (which is what a directory is), but as a logical unit.\nIf I bind another project 'frotz' at my path 'lib/frotz' and ask for\ndiff of two versions of my superproject, e.g.\n\n $ git diff v1.0 v1.1\n\nI am more interested in finding out that we used to use v2.5 of\n'frotz' back when we were at v1.0 but in our v1.1 we downgraded it\nto v2.4, perhaps because we found a regression in the 'frotz'\nlibrary, rather than the whole reverse differences between v2.4 and\nv2.5 of the frotz project. That difference, when I want to, I can\nget by going to that submodule and grab it myself with\n\n $ cd lib/frotz &7 git diff v2.4 v2.5\n\nI also suspect that you do not have to change \"git diff\" at all to\nshow the patch recursively by using the attribute mechanism (look in\nDocumentation/gitattributes.text for a string GIT_EXTERNAL_DIFF).\nIt might be just as simple as doing this:\n\n\techo >.gitattributes \"/lib/frotz diff=subrecurse\" \n\tgit config diff.subrecurse.command $HOME/bin/diff-subrecurse\n cat >$HOME/bin/diff-subrecurse <<\\-EOF\n\t#!/bin/sh\n path=$1 old_hex=$3 new_hex=$6\n unset GIT_DIR\n cd \"$path\" || exit 1\n git diff \"$old_hex\" \"$new_hex\" \n EOF\n chmod +x $HOME/bin/diff-subrecurse\n\nThe corner cases like \"new submodule\", \"removed submodule\" are left\nas an exercise to the reader ;-)\n"},{"id":"199541","messageId":"7v4nmtfxvx.fsf@alter.siamese.dyndns.org","threadId":"31580","inReplyTo":"7vobl3m8pb.fsf@alter.siamese.dyndns.org","subject":"Re: git diff across submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-09-19T22:31:14Z","receivedAt":"2012-09-19T22:31:14Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> I also suspect that you do not have to change \"git diff\" at all to\n> show the patch recursively by using the attribute mechanism (look in\n> Documentation/gitattributes.text for a string GIT_EXTERNAL_DIFF).\n> It might be just as simple as doing this:\n>\n> \techo >.gitattributes \"/lib/frotz diff=subrecurse\" \n> \tgit config diff.subrecurse.command $HOME/bin/diff-subrecurse\n> cat >$HOME/bin/diff-subrecurse <<\\-EOF\n> \t#!/bin/sh\n> path=$1 old_hex=$3 new_hex=$6\n> unset GIT_DIR\n> cd \"$path\" || exit 1\n> git diff \"$old_hex\" \"$new_hex\" \n> EOF\n> chmod +x $HOME/bin/diff-subrecurse\n>\n> The corner cases like \"new submodule\", \"removed submodule\" are left\n> as an exercise to the reader ;-)\n\nIt turns out that essentially the above outline I concocted in my\nMUA is usable almost as-is.\n\nHere is what I ended up with.\n\n * In .git/config of the superproject, I added this:\n\n [diff \"submodule-recurse\"]\n\t\tcommand = src/bin/diff-submodule-recurse\n\n * In the superproject, src/bin/diff-submodule-recurse has this\n (this is probably whitespace damaged---the lines must be indented\n by HT for the here document to correctly work):\n\n #!/bin/sh\n # $1 $2 $3 $4 $5 $6 $7\n # path old-file old-hex old-mode new-file new-hex new-mode\n\n case \"$#,$4,$7\" in\n 7,160000,160000) ;;\n *) echo \"diff --git a/$1 b/$1\"\n echo \"(punt)\"\n exit\n ;;\n esac\n unset GIT_DIR\n cd \"$1\" || {\n cat <<-\\EOF\n diff --git a/$1 b/$1\n (cannot chdir to $1)\n\t\t-Subproject commit $3\n\t\t+Subproject commit $6\n EOF\n }\n git --no-pager diff --src-prefix=\"s/$1/\" --dst-prefix=\"m/$1/\" \"$3\" \"$6\"\n\n * In .gitattributes of the superproject, I have this:\n\n /var\tdiff=submodule-recurse\n\nThe superproject in this case is a repository to control what I have\nin my $HOME directory (e.g. it has src/dot/Makefile that builds and\ninstalls the appropriate dotfiles, src/bin/Makefile that builds and\ninstalls to $HOME/bin, etc.), and one subdirectory, 'var', is a\nsubmodule that is only cloned to some but not all machines I clone\nthis superproject to.\n\nWith this setting, things like\n\n\t$ git diff HEAD~20\n\nshow differences with recursion into the var/ submodule just fine.\n"},{"id":"199622","messageId":"505B73CE.5030009@web.de","threadId":"31580","inReplyTo":"7v4nmtfxvx.fsf@alter.siamese.dyndns.org","subject":"Re: git diff across submodules","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2012-09-20T19:51:42Z","receivedAt":"2012-09-20T19:51:42Z","isPatch":false,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 20.09.2012 00:31, schrieb Junio C Hamano:\n> Junio C Hamano writes:\n> \n>> I also suspect that you do not have to change \"git diff\" at all to\n>> show the patch recursively by using the attribute mechanism (look in\n>> Documentation/gitattributes.text for a string GIT_EXTERNAL_DIFF).\n>> It might be just as simple as doing this:\n>>\n>> \techo >.gitattributes \"/lib/frotz diff=subrecurse\" \n>> \tgit config diff.subrecurse.command $HOME/bin/diff-subrecurse\n>> cat >$HOME/bin/diff-subrecurse <<\\-EOF\n>> \t#!/bin/sh\n>> path=$1 old_hex=$3 new_hex=$6\n>> unset GIT_DIR\n>> cd \"$path\" || exit 1\n>> git diff \"$old_hex\" \"$new_hex\" \n>> EOF\n>> chmod +x $HOME/bin/diff-subrecurse\n>>\n>> The corner cases like \"new submodule\", \"removed submodule\" are left\n>> as an exercise to the reader ;-)\n> \n> It turns out that essentially the above outline I concocted in my\n> MUA is usable almost as-is.\n> \n> Here is what I ended up with.\n> \n> * In .git/config of the superproject, I added this:\n> \n> [diff \"submodule-recurse\"]\n> \t\tcommand = src/bin/diff-submodule-recurse\n> \n> * In the superproject, src/bin/diff-submodule-recurse has this\n> (this is probably whitespace damaged---the lines must be indented\n> by HT for the here document to correctly work):\n> \n> #!/bin/sh\n> # $1 $2 $3 $4 $5 $6 $7\n> # path old-file old-hex old-mode new-file new-hex new-mode\n> \n> case \"$#,$4,$7\" in\n> 7,160000,160000) ;;\n> *) echo \"diff --git a/$1 b/$1\"\n> echo \"(punt)\"\n> exit\n> ;;\n> esac\n> unset GIT_DIR\n> cd \"$1\" || {\n> cat <<-\\EOF\n> diff --git a/$1 b/$1\n> (cannot chdir to $1)\n> \t\t-Subproject commit $3\n> \t\t+Subproject commit $6\n> EOF\n> }\n> git --no-pager diff --src-prefix=\"s/$1/\" --dst-prefix=\"m/$1/\" \"$3\" \"$6\"\n> \n> * In .gitattributes of the superproject, I have this:\n> \n> /var\tdiff=submodule-recurse\n> \n> The superproject in this case is a repository to control what I have\n> in my $HOME directory (e.g. it has src/dot/Makefile that builds and\n> installs the appropriate dotfiles, src/bin/Makefile that builds and\n> installs to $HOME/bin, etc.), and one subdirectory, 'var', is a\n> submodule that is only cloned to some but not all machines I clone\n> this superproject to.\n> \n> With this setting, things like\n> \n> \t$ git diff HEAD~20\n> \n> show differences with recursion into the var/ submodule just fine.\n\nThat's pretty cool! Even though diff options like --stat and --name-only\nstill won't take into account what happened inside the submodule this\napproach makes it possible to see the diff recursively. Wouldn't it make\nsense add this script to contrib (after teaching it new and removed\nsubmodules and documenting its use in a few lines after the shebang)?\n"},{"id":"199626","messageId":"7vr4pwa02u.fsf@alter.siamese.dyndns.org","threadId":"31580","inReplyTo":"505B73CE.5030009@web.de","subject":"Re: git diff across submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-09-20T20:52:41Z","receivedAt":"2012-09-20T20:52:41Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jens Lehmann writes:\n\n> That's pretty cool! Even though diff options like --stat and --name-only\n> still won't take into account what happened inside the submodule this\n> approach makes it possible to see the diff recursively. Wouldn't it make\n> sense add this script to contrib (after teaching it new and removed\n> submodules and documenting its use in a few lines after the shebang)?\n\nA few things somebody may want to work on while doing that \"few\nlines of documentation\" I know about are:\n\n * From the core side, pass options that are releavant when\n generating patch (i.e. with p) in environment variables to the\n external diff script;\n\n * Not using \"s/$1\" and \"m/$1\" as prefix; instead, pass src/dst\n prefix values (i.e. s/ and m/) from the core side in environment\n variables, and make the external diff script itself aware of\n possibly nested submodules, e.g.\n\n SUBMODULE_PATH=\"${SUBMODULE_PATH}$1\"\n export SUBMODULE_PATH\n exec git --no-pager diff -p \\\n \t--src-prefix=\"$SRC_PREFIX/$SUBMODULE_PATH\" \\\n \t--dst-prefix=\"$DST_PREFIX/$SUBMODULE_PATH\" \"$3\" \"$6\"\n\nAfter people gain sufficient experience with it, as the next step,\nwe can think about how to handle --stat and other options when we\nare run without -p (currently the attribute mechanism would not\ntrigger) and then we can call the result a native \"diff that\nrecurses into submodules\" that people can use without setting up the\nattributes based mechanism.\n"}]}