{"thread":{"id":"28798","subject":"[PATCH] Display change history as a diff between two dirs","startedAt":"2011-10-29T20:51:49Z","lastAt":"2011-11-01T17:15:55Z","messageCount":6,"participants":["Roland Kaufmann","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"178515","messageId":"4EAC6765.4030003@gmail.com","threadId":"28798","inReplyTo":null,"subject":"[PATCH] Display change history as a diff between two dirs","fromName":"Roland Kaufmann","fromEmail":"rlndkfmn+git@gmail.com","sentAt":"2011-10-29T20:51:49Z","receivedAt":"2011-10-29T20:51:49Z","isPatch":true,"sender":{"key":"rlndkfmn+git@gmail.com","avatar":null},"body":"Watching patches serially it can be difficult to get an overview of how\na pervasive change is distributed through-out different modules. Thus:\n\nExtract snapshots of the files that have changed between two revisions\ninto temporary directories and launch a graphical tool to show the diff\nbetween them.\n\nUse existing functionality in git-diff to figure out which files have\nchanged, and to get the files themselves.\n\nBased on a script called 'git-diffc' by Nitin Gupta.\n\nSigned-off-by: Roland Kaufmann <rlndkfmn+git@gmail.com>\n---\n\nRequests for such a scripts surface occationally, so I believe it could\nbe useful to have in the distribution itself.\n\n Documentation/git-dirdiff.txt |   55 +++++++++++++++++++++++++++++++++++++++++\n Makefile                      |    2 +\n git-dirdiff--helper.sh        |   28 +++++++++++++++++++++\n git-dirdiff.sh                |   34 +++++++++++++++++++++++++\n 4 files changed, 119 insertions(+), 0 deletions(-)\n create mode 100644 Documentation/git-dirdiff.txt\n create mode 100755 git-dirdiff--helper.sh\n create mode 100755 git-dirdiff.sh\n\ndiff --git a/Documentation/git-dirdiff.txt b/Documentation/git-dirdiff.txt\nnew file mode 100644\nindex 0000000..bdd2581\n--- /dev/null\n+++ b/Documentation/git-dirdiff.txt\n@@ -0,0 +1,55 @@\n+git-dirdiff(1)\n+==============\n+\n+NAME\n+----\n+git-dirdiff - Show changes using directory compare\n+\n+SYNOPSIS\n+--------\n+[verse]\n+'git dirdiff' [<options>] [<commit> [<commit>]] [--] [<path>...]\n+\n+DESCRIPTION\n+-----------\n+'git dirdiff' is a git command that allows you to compare revisions\n+as a difference between two directories. 'git dirdiff' is a frontend\n+to linkgit:git-diff[1].\n+\n+OPTIONS\n+-------\n+See linkgit:git-diff[1] for the list of supported options.\n+\n+CONFIG VARIABLES\n+----------------\n+'git dirdiff' uses the same config variables as linkgit:git-difftool[1]\n+to determine which difftool should be used.\n+\n+TEMPORARY FILES\n+---------------\n+'git dirdiff' creates a directory with 'mktemp' to hold snapshots of the\n+files which are different in the two revisions. This directory is removed\n+when the diff viewer terminates.\n+\n+NOTES\n+-----\n+The diff viewer must support being passed directories instead of files\n+as its arguments.\n++\n+Files that are not put under version control are not included when\n+viewing the difference between a revision and the working directory.\n+\n+SEE ALSO\n+--------\n+linkgit:git-diff[1]::\n+\t Show changes between commits, commit and working tree, etc\n+\n+linkgit:git-difftool[1]::\n+\tShow changes using common diff tools\n+\n+linkgit:git-config[1]::\n+\t Get and set repository or global options\n+\n+GIT\n+---\n+Part of the linkgit:git[1] suite\ndiff --git a/Makefile b/Makefile\nindex 3139c19..03771cf 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -365,6 +365,8 @@ unexport CDPATH\n SCRIPT_SH += git-am.sh\n SCRIPT_SH += git-bisect.sh\n SCRIPT_SH += git-difftool--helper.sh\n+SCRIPT_SH += git-dirdiff.sh\n+SCRIPT_SH += git-dirdiff--helper.sh\n SCRIPT_SH += git-filter-branch.sh\n SCRIPT_SH += git-lost-found.sh\n SCRIPT_SH += git-merge-octopus.sh\ndiff --git a/git-dirdiff--helper.sh b/git-dirdiff--helper.sh\nnew file mode 100755\nindex 0000000..bc0b49d\n--- /dev/null\n+++ b/git-dirdiff--helper.sh\n@@ -0,0 +1,28 @@\n+#!/bin/sh\n+#\n+# Accumulate files in a changeset into a pre-defined directory.\n+#\n+# Copyright (C) 2011 Roland Kaufmann\n+# Based on a script called git-diffc by Nitin Gupta\n+#\n+# This file is licensed under the GPL v2, or a later version\n+# at the discretion of the official Git maintainer.\n+\n+# bail out if there is any problems copying\n+set -e\n+\n+# check that we are called by git-dirdiff\n+if [ -z $__GIT_DIFF_DIR ]; then\n+  echo Error: Do not call $(basename $0) directly 1>&2\n+  exit 1\n+fi\n+\n+# don't attempt to copy new or removed files\n+if [ \"$2\" != \"/dev/null\" ]; then\n+  mkdir -p $__GIT_DIFF_DIR/old/$(dirname $1)\n+  cp $2 $__GIT_DIFF_DIR/old/$1\n+fi\n+if [ \"$5\" != \"/dev/null\" ]; then\n+  mkdir -p $__GIT_DIFF_DIR/new/$(dirname $1)\n+  cp $5 $__GIT_DIFF_DIR/new/$1\n+fi\ndiff --git a/git-dirdiff.sh b/git-dirdiff.sh\nnew file mode 100755\nindex 0000000..4e75eda\n--- /dev/null\n+++ b/git-dirdiff.sh\n@@ -0,0 +1,34 @@\n+#!/bin/sh\n+#\n+# Display differences between two commits with a directory comparison.\n+#\n+# Copyright (C) 2011 Roland Kaufmann\n+# Based on a script called git-diffc by Nitin Gupta\n+#\n+# This file is licensed under the GPL v2, or a later version\n+# at the discretion of the official Git maintainer.\n+\n+# bail out if there is any problems in getting a diff\n+set -e\n+\n+# create a temporary directory to hold snapshots of changed files\n+__GIT_DIFF_DIR=$(mktemp --tmpdir -d git-dirdiff.XXXXXX)\n+export __GIT_DIFF_DIR\n+\n+# cleanup after we're done\n+trap 'rm -rf $__GIT_DIFF_DIR' 0\n+\n+# list all files that have changed. store this list in a separate\n+# file so that we can test the exit status of this command. (if we had\n+# bash we could use pipefail, or if we had Posix we could use mkfifo)\n+git diff --raw \"$@\" > $__GIT_DIFF_DIR/toc\n+\n+# let the helper script accumulate them into the temporary directory\n+cut -f 2- -s $__GIT_DIFF_DIR/toc | while read f; do\n+  GIT_EXTERNAL_DIFF=git-dirdiff--helper git --no-pager diff \"$@\" $f\n+done\n+\n+# run original diff program, reckoning it will understand directories\n+# modes and shas does not apply to the root directories so submit dummy\n+# values for those, hoping that the diff tool does not use them.\n+git-difftool--helper - $__GIT_DIFF_DIR/old deadbeef 0755 $__GIT_DIFF_DIR/new babeface 0755\n-- \n1.7.1\n"},{"id":"178519","messageId":"7vty6rrow8.fsf@alter.siamese.dyndns.org","threadId":"28798","inReplyTo":"4EAC6765.4030003@gmail.com","subject":"Re: [PATCH] Display change history as a diff between two dirs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-10-30T06:09:59Z","receivedAt":"2011-10-30T06:09:59Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Roland Kaufmann <rlndkfmn+git@gmail.com> writes:\n\n> diff --git a/git-dirdiff--helper.sh b/git-dirdiff--helper.sh\n> new file mode 100755\n> index 0000000..bc0b49d\n> --- /dev/null\n> +++ b/git-dirdiff--helper.sh\n> @@ -0,0 +1,28 @@\n> ...\n> +# bail out if there is any problems copying\n> +set -e\n\nI do not think any of our scripted Porcelains use \"set -e\"; rather they\ntry to catch errors and produce messages that are more appropriate for\nspecific error sites.\n\n> +# check that we are called by git-dirdiff\n> +if [ -z $__GIT_DIFF_DIR ]; then\n> +  echo Error: Do not call $(basename $0) directly 1>&2\n> +  exit 1\n> +fi\n\n(style)\n\n\tif test -z \"$__GIT_DIFF_DIR\"\n        then\n        \techo >&2 \"...\"\n\t\texit 1\n\tfi\n\nIf this were to become part of the main Porcelain set, you would probably\nsource in the git-sh-setup helper near the beginning with\n\n\t. git-sh-setup\n\nand use \"die\" instead of \"echo && exit 1\".\n\n> diff --git a/git-dirdiff.sh b/git-dirdiff.sh\n> new file mode 100755\n> index 0000000..4e75eda\n> --- /dev/null\n> +++ b/git-dirdiff.sh\n> @@ -0,0 +1,34 @@\n> ...\n> +# create a temporary directory to hold snapshots of changed files\n> +__GIT_DIFF_DIR=$(mktemp --tmpdir -d git-dirdiff.XXXXXX)\n> +export __GIT_DIFF_DIR\n\nI do not think any of our scripted Porcelains use \"mktemp\" especially\n\"mktemp -d\". How portable is it?\n\n> +git diff --raw \"$@\" > $__GIT_DIFF_DIR/toc\n> +\n> +# let the helper script accumulate them into the temporary directory\n> +cut -f 2- -s $__GIT_DIFF_DIR/toc | while read f; do\n> +  GIT_EXTERNAL_DIFF=git-dirdiff--helper git --no-pager diff \"$@\" $f\n> +done\n\n(style)\n\n\t... upstream command ... |\n        while read f\n\tdo\n\t\t...\n\tdone\n\nIt also is not clear what could be used in \"$@\". Obviously you would not\nwant things like \"-U20\" and \"--stat\" fed to the first \"list of paths\"\nphase, but there may be some options you may want to give to the inner\n\"external diff\" thing.\n\nIt is not clear to me why you need to grab the list of paths in toc and\niterate over them one by one. IOW, why isn't this sufficient?\n\n    # dirdiff--helper will copy the files in the temporary directory\n    GIT_EXTERNAL_DIFF=git-dirdiff--helper git --no-pager diff \"$@\"\n\n    # Now the temporary old/ and new/ are populated, compare them\n    git-difftool--helper - .....\n\nOverall, I found it interesting, but I am not convinced yet that this\nshould be in the set of main Porcelains.  It seems to be a hack to work\naround the design of the current external diff interface that specifically\ntargets tools that are about comparing single pair of paths at a time.\n\nIf \"compare two sets of files, each extracted in its own temporary\ndirectory\" turns out to be sufficiently useful thing to do (which I\nsuspect is true), we would probably want to make it an option to \"git\ndiff\" itself, and not a separate git subcommand like \"dirdiff\". I can see\n\"git diff\" (and possibly even things like \"git log -p\") populating two\ntemporary directories (your old/ and new/) and running a custom program\nspecified by GIT_EXTERNAL_TREEDIFF environment variable, instead of doing\nany textual diff generation internally.\n\nI wouldn't mind carrying a polished version of this in contrib/ for a\ncycle or two in order to let people try it out and get kinks out of its\ndesign.\n\nFor example, how well does this approach work with -M/-C (I do not think\nit would do anything useful, but I haven't thought things through).  It\nwould be nice if we gave the hint to the external program that compares\nthe populated temporary directories how paths in the preimage tree and\npostimage tree correspond to each other.\n"},{"id":"178557","messageId":"4EAE688C.8010502@gmail.com","threadId":"28798","inReplyTo":"7vty6rrow8.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] Display change history as a diff between two dirs","fromName":"Roland Kaufmann","fromEmail":"rlndkfmn+git@gmail.com","sentAt":"2011-10-31T09:21:16Z","receivedAt":"2011-10-31T09:21:16Z","isPatch":true,"sender":{"key":"rlndkfmn+git@gmail.com","avatar":null},"body":"On 2011-10-30 07:09, Junio C Hamano wrote:\n> I do not think any of our scripted Porcelains use \"set -e\"; rather\n> they try to catch errors and produce messages that are more\n> appropriate for specific error sites.\n\ngit-dirdiff being a wrapper script, I reasoned that the underlaying\ncommand would already have printed a sufficient error message, so what\nwas left was just to exit. But you're right in that some of the commands\nwill not give sufficient context. I'll put in some more detailed error\nhandling.\n\n> I do not think any of our scripted Porcelains use \"mktemp\"\n> especially \"mktemp -d\". How portable is it?\n\nEven if it is not part of Posix, I reckon since it is a part of the\ncoreutils, it is available in most GNU userlands, inclusive GnuWin32.\nI don't have any live BSD systems available, but based on the man pages\nit seems to be available there too, albeit with some different options.\n\n--tmpdir seems to be more of a problem in than respect than -d. I'll see\nif I can find a set of options that are digestable to most platforms.\n\nI think it is unfortunate to use the current directory as scratch space;\nit can be a network disk, or even read-only. mktemp can in contrast make\na directory in a place which is not spilled to disk.\n\n> It is not clear to me why you need to grab the list of paths in toc\n> and iterate over them one by one. IOW, why isn't this sufficient?\n\nThis design decision was probably appropriate in an earlier iteration,\nbut as you point out, it is indeed superfluous now! I apologize for not\nrealizing that myself.\n\n> suspect is true), we would probably want to make it an option to\n> \"git diff\" itself, and not a separate git subcommand like \"dirdiff\".\n> I can see\n\nAlthough being an able *user* of Git, I am not (yet) familiar enough\nwith the codebase to have a patch ready for `git diff` itself. It would\ncertainly require more rigorous testing.\n\n> \"git diff\" (and possibly even things like \"git log -p\") populating\n> two temporary directories (your old/ and new/) and running a custom\n> program specified by GIT_EXTERNAL_TREEDIFF environment variable,\n> instead of doing\n\nWould it be better to have yet another configuration available for this \noption instead of reusing the existing infrastructure for `git difftool`?\n\n> It also is not clear what could be used in \"$@\". Obviously you would\n>  not want things like \"-U20\" and \"--stat\" fed to the first \"list of\n> paths\" phase, but there may be some options you may want to give to\n> the inner \"external diff\" thing.\n\nIdeally, it should work the same way as `git difftool`.\n\n> For example, how well does this approach work with -M/-C (I do not\n> think it would do anything useful, but I haven't thought things\n> through).  It would be nice if we gave the hint to the external\n\nAs of now, files that are moved will turn up a different places in the\ntree without any annotations, and off the top of my head I don't see any\nobvious way to propagate such hints. If the diff tool is sophisticated\ncan probably use the same heuristics as Git currently does, but I am\nunaware of any that is able to do that yet.\n\n> I wouldn't mind carrying a polished version of this in contrib/ for\n> a cycle or two in order to let people try it out and get kinks out of\n>  its design.\n\nI would appreciate that! I'll submit a reworked proposal taking you\ncomments into account. It may take a few days due to unrelated\nengagements, though.\n\n-- \n   Roland.\n"},{"id":"178610","messageId":"7v39e8o61s.fsf@alter.siamese.dyndns.org","threadId":"28798","inReplyTo":"4EAE688C.8010502@gmail.com","subject":"Re: [PATCH] Display change history as a diff between two dirs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-11-01T03:49:51Z","receivedAt":"2011-11-01T03:49:51Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Roland Kaufmann <rlndkfmn+git@gmail.com> writes:\n\n>> \"git diff\" (and possibly even things like \"git log -p\") populating\n>> two temporary directories (your old/ and new/) and running a custom\n>> program specified by GIT_EXTERNAL_TREEDIFF environment variable,\n>> instead of doing\n>\n> Would it be better to have yet another configuration available for\n> this option instead of reusing the existing infrastructure for `git\n> difftool`?\n\nGIT_EXTERNAL_DIFF is designed to drive tools that can take a single pair\nof files and compare, so it is entirely possible for people to have such\ntool that would _not_ grok (as a tool named by the variable does not have\nto) two directories specified by it. So yes, it is essential that the\nvariable not be reused.\n\nIt probably is OK for \"git diff --dirdiff\" to use GIT_EXTERNAL_TREEDIFF if\nand only if GIT_EXTERNAL_DIFF is not defined, and use GIT_EXTERNAL_DIFF\notherwise. People who have GIT_EXTERNAL_DIFF set to a tool capable of\nhanding directory pair can just add \"--dirdiff\" to the command line, and\nothers can find such a tool and set it to GIT_EXTERNAL_TREEDIFF when they\ndo so.\n\n>> It also is not clear what could be used in \"$@\". Obviously you would\n>>  not want things like \"-U20\" and \"--stat\" fed to the first \"list of\n>> paths\" phase, but there may be some options you may want to give to\n>> the inner \"external diff\" thing.\n>\n> Ideally, it should work the same way as `git difftool`.\n\nI am not so sure about that; difftool is another way to drive a tool one\nfilepair at a time. You want to drive a tool that takes them as two _sets_\nof files, one in each directory.\n"},{"id":"178617","messageId":"CANEMaQRoXRygmrYsu0xF6mWq50r9Qhh-YuXXNAhyQewd+fDtsQ@mail.gmail.com","threadId":"28798","inReplyTo":"7v39e8o61s.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] Display change history as a diff between two dirs","fromName":"Roland Kaufmann","fromEmail":"rlndkfmn+git@gmail.com","sentAt":"2011-11-01T09:01:33Z","receivedAt":"2011-11-01T09:01:33Z","isPatch":true,"sender":{"key":"rlndkfmn+git@gmail.com","avatar":null},"body":"On Tue, Nov 1, 2011 at 04:49, Junio C Hamano <gitster@pobox.com> wrote:\n> Roland Kaufmann <rlndkfmn+git@gmail.com> writes:\n>> Would it be better to have yet another configuration available for\n>> this option instead of reusing the existing infrastructure for `git\n>> difftool`?\n\n> It probably is OK for \"git diff --dirdiff\" to use GIT_EXTERNAL_TREEDIFF if\n> and only if GIT_EXTERNAL_DIFF is not defined, and use GIT_EXTERNAL_DIFF\n> otherwise. People who have GIT_EXTERNAL_DIFF set to a tool capable of\n> handing directory pair can just add \"--dirdiff\" to the command line, and\n\nDid you perhaps mean the other way around: GIT_EXTERNAL_TREEDIFF\nif set, and GIT_EXTERNAL_DIFF otherwise?\n\nI will revise the patch to incorporate that, and at the same time fix\nthe isempty\nfunction (it can be done simpler).\n\n>>> It also is not clear what could be used in \"$@\". Obviously you would\n>>>  not want things like \"-U20\" and \"--stat\" fed to the first \"list of\n>>> paths\" phase, but there may be some options you may want to give to\n>>> the inner \"external diff\" thing.\n\n>> Ideally, it should work the same way as `git difftool`.\n\n> I am not so sure about that; difftool is another way to drive a tool one\n> filepair at a time. You want to drive a tool that takes them as two _sets_\n> of files, one in each directory.\n\nI was thinking about the behaviour when you send options that are not\napplicable to files at all, such as \"--stat\". In that case, it just stops after\n`git diff`. Options that only applies to one file and not to a set\nhave I admittedly\nnot thought about (yet).\n\n-- \n   Roland.\n"},{"id":"178632","messageId":"7vr51rn4qc.fsf@alter.siamese.dyndns.org","threadId":"28798","inReplyTo":"CANEMaQRoXRygmrYsu0xF6mWq50r9Qhh-YuXXNAhyQewd+fDtsQ@mail.gmail.com","subject":"Re: [PATCH] Display change history as a diff between two dirs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-11-01T17:15:55Z","receivedAt":"2011-11-01T17:15:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Roland Kaufmann <rlndkfmn+git@gmail.com> writes:\n\n> On Tue, Nov 1, 2011 at 04:49, Junio C Hamano <gitster@pobox.com> wrote:\n>> Roland Kaufmann <rlndkfmn+git@gmail.com> writes:\n>>> Would it be better to have yet another configuration available for\n>>> this option instead of reusing the existing infrastructure for `git\n>>> difftool`?\n>\n>> It probably is OK for \"git diff --dirdiff\" to use GIT_EXTERNAL_TREEDIFF if\n>> and only if GIT_EXTERNAL_DIFF is not defined, and use GIT_EXTERNAL_DIFF\n>> otherwise. People who have GIT_EXTERNAL_DIFF set to a tool capable of\n>> handing directory pair can just add \"--dirdiff\" to the command line, and\n>\n> Did you perhaps mean the other way around: GIT_EXTERNAL_TREEDIFF\n> if set, and GIT_EXTERNAL_DIFF otherwise?\n\nI think I misstated the precedence order.  What I had in mind was roughly\nlike this:\n\n 1. \"git diff\" (and others like \"git log\") without \"--dirdiff\" option:\n    GIT_EXTERNAL_DIFF is used just as before and no GIT_EXTERNAL_TREEDIFF\n    is consulted;\n\n 2. \"git diff --dirdiff\": GIT_EXTERNAL_TREEDIFF is used if set, otherwise\n    GIT_EXTERNAL_DIFF is used.\n\nThat way, people who have been using GIT_EXTERNAL_DIFF that is capable of\ncomparing two directories do not have to change anything and can just give\n\"--dirdiff\" when they want to operate the command with this new mode of\ncomparison. People whose GIT_EXTERNAL_DIFF is not capable of comparing two\ndirectories have two choices, but in either case they first would need to\nfind an external tool they want to use with the \"--dirdiff\" mode. If they\nwant to use the same new external tool for non \"--dirdiff\" mode, then they\ncan set GIT_EXTERNAL_DIFF to it. If they want to keep using the command\nthey have been using with GIT_EXTERNAL_DIFF when not in \"--dirdiff\" mode,\nhowever, they can keep their GIT_EXTERNAL_DIFF to the old \"two files\ncomparison\" tool, and set GIT_EXTERNAL_TREEDIFF to the new one.\n\nBut that is all about envisioning how a proper integration of the feature\nin the main \"diff/log -p\" command codepath would work. I do not know if it\nis worth to add that to your script, which would be a stop-gap experiment\nuntil we find out if the \"populate two temporary directories and have a\ntool compare them as a whole\" mode is useful.\n"}]}