patchdifftool: avoid symlinks when reusing worktree files

difftool's dir-diff should never reuse a symlink, regardless of what it points to. Tighten use_wt_file() so that it rejects all symlinks.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: David Aguilar <davvid@gmail.com>
---
 git-difftool.perl | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

David Aguilar <davvid@gmail.com> writes:

Sorry. I do recall saying "it is wrong to feed the contents of a file that a symlink points at to hash-object" but other than that, I completely lost track.

What purpose does this function play in its callchain? What does its caller wants it to compute? Is use of the entity in the working tree completely optional? Would the caller happily produce correct result even if we changed this function to unconditionally return ($use=0, $wt_sha1='0'x40) regardless of the result of lstat(2) on "$workdir/$file"?

The conclusion of the thought process that starts from "it is wrong to feed the contents of a file that a symlink points at to hash-object" may not be "so let's return $use=0 for all symlinks", which is this patch. Depending on what its caller wants it to compute, the right conclusion may be "we need to call hash-object correctly by first running readlink and then feeding the result to it".

And if the answer is "the caller wants us to compute the hash for a symbolic link and say $use=1", then we would instead need to do an equivalent of

	wt_sha1=$(readlink "$workdir/$file" | hash-object --stdin)

I cannot quite tell which from the patch and explanation.

Perhaps an additional test or two would help illustrate what issues are being addressed better?

Thanks.

On Tue, Oct 27, 2015 at 03:24:49PM -0700, Junio C Hamano wrote:

Right. At first I thought I could revise the commit message to make it clearer that we simply want to skip all symlinks, since it never makes sense to reuse a worktree symlinks, but looking at the tests and implementation makes me realize that it's not that simple.

This is going to take a bit more time to get right. John, I was hoping you'd be able to take a look -- I'm playing catch-up too. When it was first reported I let it sit for a while in hopes that the original author would pickup the issue, but months passed and I figured I'd take a stab at helping the user out.

Anyways, it'll take me a bit more time to understand the code and work out a sensible solution. My gut feeling is that we should adjust the dir-diff feature so that it ignores all symlinks. That seems like a simple answer since we're deciding to skip that chunk of complexity.

John, do you have any thoughts on how we can best handle this?

David Aguilar <davvid@gmail.com> writes:

What dir-diff wants to do is to prepare two directory hierarchies on the filesystem and run "diff -r" (or an equivalent command of user's choice) on them, e.g. "diff -r left/ right/". "left/" tree is typically what you want to compare your working tree files agaist (e.g. a clean checkout of "the other version"), and "right/" tree is populated with either copies of the working tree or symbolic links. The copying to "right/" feels wasteful, but your working tree may be littered with build artifacts, and making a clean copy with only tracked files is one way to make sure that "diff -r" with a clean checout of "the other version" will not show them.

In the loop that walks the @rawdiff array, there are a lot of "if we see a symbolic link on the left, do this" before the last one that says "if the working tree side is not $null (i.e. not missing), ask ut_wt_file()". That code remembers which path on either side had symbolic links.

Later in the same function, there is this comment "Symbolic links require special treatment." The intent of the code here is that any path that involves a symbolic link should be tweaked there. The loop over %symlink expects left/ and right/ to be populated normally by the loop over @working_tree, and then for any path that is a symbolic link is replaced with a phony regular file (not a symbolic link) that says "Here is a symbolic link".

So I think it is fine to return $use=0 for any symbolic link from use_wt_file. Anything you do there will be replaced by the loop over %symlink that appears later in the caller. The caller discards $wt_sha1 when $use=0 is returned, so the second return value does not matter.

Junio C Hamano <gitster@pobox.com> writes:

So let me try to update your patch with the result of the study of the codeflow.

-- >8 --
From: David Aguilar <davvid@gmail.com>
Subject: difftool: ignore symbolic links in use_wt_file

The caller is preparing a narrowed-down copy of the working tree and this function is asked if the path should be included in that copy. If we say yes, the path from the working tree will be either symlinked or copied into the narrowed-down copy.

For any path that is a symbolic link, the caller later fixes up the narrowed-down copy by unlinking the path and replacing it with a regular file it writes out that mimics the way how "git diff" compares symbolic links.

Let's answer "no, you do not want to copy/symlink the working tree file" for all symbolic links from this function, as we know the result will not be used because it will be overwritten anyway.

Incidentally, this also stops the function from feeding a symbolic link in the working tree to hash-object, which is a wrong thing to do to begin with. The link may be pointing at a directory, or worse may be dangling (both would be noticed as an error). Even if the link points at a regular file, hashing the contents of a file that is pointed at by the link is not correct (Git hashes the contents of the link itself, not the pointee).

Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 git-difftool.perl   |  4 +---
 t/t7800-difftool.sh | 19 +++++++++++++++++++
 2 files changed, 20 insertions(+), 3 deletions(-)

On Thu, Oct 29, 2015 at 11:19:01AM -0700, Junio C Hamano wrote:

This is a very nicely worded commit message. Thanks for the thorough explanation.

The "-e _" shorthand caught my eye ~ I didn't know perl could do that! Nice.

Underline is barely mentioned in perlvar, but it's obvious what (I think) it means, and since Perl is DWIM, it must be right. ;-)

David Aguilar <davvid@gmail.com> writes:

It is mentioned under -X (i.e. the headline for -f, -e, -d, ... tests) in perlfunc with an interesting example of doing an explicit stat() first and then running many variants of -X with _ as the target.

David Aguilar <davvid@gmail.com> writes:

I still don't know if return (0, $null) is the right thing to do, but in any case, I find the original flow easier to read, i.e. "we notice a few cases we cannot do the main 'hash-object' thing this function is meant to do and return early". I.e.

	if (-l "$workdir/$file" || ! -f _) {
        	return (0, $null_sha1);
	}
	my $wt_sha1 = $repo->command_oneline('hash-object', "$workdir/$file");
	my $use = ($sha1 eq $null_sha1) || ($sha1 eq $wt_sha1);
	return ($use, $wt_sha1);