Help understanding unexpected diff output

Hi!

I hope we all agree that diff should make our lives easier, so here is something where I think it does not apply:

I inspected some changes using "git diff", and it was hard to me to find the actual differences (see git-diff-png): I had to look a long time at those olcAccess lines to find any difference, so finally I pasted both blocks into separate Emacs files a and b (via PuTTY) and used "ediff-files" (see diff-emacs.png). Within there, the was only a part of one line changed!

I don't understand this. I had even used "git diff -w" to ignore space changes (which aren't expected anyway).

Any ideas what might be wrong and how to get better diff output?

(git-2.26.2-27.36.1.x86_64 of SLES12 being used)

Regards, Ulrich

"Ulrich Windl" <Ulrich.Windl@rz.uni-regensburg.de> writes:

With redactions and making them into two images, you did a good job killing interested parties' willingness to help, as they made using tools almost impossible.

So here is only based on eyeballing the images, and I probably have missed what you wanted readers to find, but don't blame me ;-)

It seems that the file got 6 lines removed, that begins with "olcAccess: {$n}" (0 <= n <= 6), at around 8th line of the file, and then very similar 6 lines that are not identical to what got removed at the end of the file. The ediff you showed tells me that the 6 lines removed from very early part of the file and 6 lines added to the end of the file are similar but not identical. These two 6-line blocks differ only by the 5th line (i.e. "olcAccess: {4}") having ",pwdGraceUseTime,pwdChangedTime" in the newer version, but not in the older version.

Am I reading what you sent us correctly so far?

Now, if this change to the "olcAccess: {4}" line in the new file weren't there, i.e. if the 6-line block that disappeared from the early part of the file and the 6-line block that was added near the end of the file were identical (in such a case, your ediff would show no changes, I presume), then what would your desired output from "git diff"?

I would imagine that the answer would still be a very similar looking

    --- a/0/...
    +++ b/0/...
    @@ -5,12 +5,6 @@
     olcDatabase: {1}hdb
     olcDbDirectory: /var/lib/ldap
     olcSuffix: dc=... 
    -olcAccess: {0} ...
    -olcAccess: {1} ...
    -olcAccess: {2} ...
    -olcAccess: {3} ...
    -olcAccess: {4} ...
    -olcAccess: {5} ...
    olcLimits: {0}dn.exact=....
     olcRootDn: cn=Admin,dc=...
     olcRootPW: {SSHA}yZ...
    @@ -52,6 +46,12 @@
     entryUUID: ...
     creatorsName: cn=config
     createTimestamp: ...
    -entryCSN: ...
    +olcAccess: {0} ...
    +olcAccess: {1} ...
    +olcAccess: {2} ...
    +olcAccess: {3} ...
    +olcAccess: {4} ...
    +olcAccess: {5} ...
    +entryCSN: ...
     modifiersName: cn=config
    -modifyTimestamp: ...
    +modifyTimestamp: ...

with identical "olcAccess: {$n}" lines removed from earlier part the file and added to the end of the file.

So I am not sure what you are asking. It would not be correct to show something like

    --- a/0/...
    +++ b/0/...
    @@ -52,6 +46,6 @@
     entryUUID: ...
     creatorsName: cn=config
     createTimestamp: ...
    -entryCSN: ...
    +entryCSN: ...
     modifiersName: cn=config
    -modifyTimestamp: ...
    +modifyTimestamp: ...

and pretend that the original "olcAccess: {$n}" lines did not get moved from the earlier part of the file to the end of the file.

Perhaps you may be looking for "git diff --color-moved"?  I dunno.

Nachricht <xmqqsgdvo3ec.fsf@gitster.c.googlers.com>:

Junio,

we had some discussions in the past, and you should know that I'm willing to provide information if needed. Part of the problem is that I can't (to my knowledge) copy and paste colored text into plain text emails. Thus screenshots. My intention was to bring up the issue, making it (more or less obvious) that the differences are much harder to see in git output than in Emacs output. The redaction was due to a password that could easily be cracked if presented and some resource name I wanted to avoid to get a clearance to publish first. It still could be a "stupid me", but can't we talk like adults?

Yes, if you look at the highlighted part (difference in cyan, "chink" in yellow) in the Emacs screenshot, it should point out the "real" differences, while git output made it harder to see.

According to my eyes (just using Emacs to make sure) those olcAccess lines "numbered" 0 to 3, and 5 should be "context lines" as they are not changed. Only line numbered {4} has a difference, and as the order 0 to 5 is unchanged I don't see why the diff-block is that large. I see that some lines that are not changed were reorderd, but that should not affect the olcAccess block. Maybe I just need different diff options; I don't know.

I kind of understand: The 5-line block of olcAccess was moved within a larger block, preventing diff from seeing the small change within. I'm attaching the "git diff --color-moved" output as another screenshot (this time with very wide terminal to make one line a line). Still I redacted ("blued-out") some information I may not publish, but believe me that the parts "blued out" are identical.

In addition I made a test-case that includes only the first 18 columns (cut -c1-18) of the file, both before (b0) and after (b1) the commit. That should be good enough for the reordering effect. A plain "diff -u shows": # diff -u b0 b1 --- b0 2020-07-14 10:43:05.000000000 +0200 +++ b1 2020-07-14 10:41:37.000000000 +0200

"Ulrich Windl" <Ulrich.Windl@rz.uni-regensburg.de> writes:

And the "slightly different" example I asked you about has {4} in both blocks identical, you are arguing that these 6 lines all should appear as context?

It certainly is possible but then the patch would look quite different. As "olcAccess: {0}" line in the preimage (i.e. a/) appears at line 8, but that same line appears in the postimage (i.e. b/) at line 49, such a patch that match these two olcAccess blocks as unchanged MUST delete many lines that come after "olcAccess: {5}" in the preimage (i.e. starting at line 14 with olcLimits, line ending at entryCSN: at line 55 must be removed, because they used to appear immediately after "olcAccess: {5}" in the preimage a/, but in the postimage b/, none of tme appear after the "olcAccess: {5}" that you claim to be common and unchanged (in the postimage, instead you have only four lines that has"entryCSN:", "modiferName", etc. before the end of the file).

Of course, these lines in the line range 14-55 actually are the ones that did not change, as we can see above, so if you insist that you must keep the 6-line "olcAccess" block as common and unchanged, because your desired patch is deleting them from the pre-image after "olcAccess: {5}" line, your desired patch must be adding them back to the postimage somewhere (namely, before "olcAccess: {0}" line).

Such a patch is also a valid one in that it expresses the difference between a/ and b/ in terms of a sequence of "delete these lines from here" and "insert these lines to here", but it would be far less interesting than what we see above. Instead of "we deleted 6 lines near the beginning of the file" plus "then added 6 lines near the end--ah, these 6 lines by the way are identical", you would instead say "we deleted the block of lines 14-55 that appear immediately after olcAccess:{5}" plus "we inserted the identical block of lines immediately before olcAccess{0}". And you'd complain in the opposite way: "These lines are identical but appear in different locations in the preimage and the postimage -- why aren't they shown as context lines?"

Hi Ulrich, (only just catching up with intriguing queries)

On 14/07/2020 09:52, Ulrich Windl wrote:

It may be me, but I think your comment above is at the core of the unexpected diff output.

If text is moved, then normally it will (obviously?) require something within the diff report to say where it was removed from, and then where it was inserted. It seems like your expectation in your use-case was that the moved line changes wouldn't be shown.

I don't believe there is an `--ignore-moved-lines` option, but if there's a good rationale / use-case for it, then maybe it's something to look at coding.

-- Philip