Re: updated design for the diff-raw format.

From: Junio C Hamano <junkio@cox.net>
Date: May 21, 2005, 23:16 UTC
Message-ID: <7vr7g0dvbj.fsf@assigned-by-dhcp.cox.net>
In-Reply-To: <7vwtpsdvgm.fsf@assigned-by-dhcp.cox.net>

(first of the replayed exchange)

To: Linus Torvalds <torvalds@osdl.org>
Subject: Re: [PATCH 3/3] Diff overhaul, adding the other half of copy
 detection.
From: Junio C Hamano <junkio@cox.net>
Date: Sat, 21 May 2005 10:56:06 -0700
Message-ID: <7v4qcwihu1.fsf@assigned-by-dhcp.cox.net>

GIT_DIFF_OPTS=--unified=0 is good to me as well (GNU diffutils 2.8.1).

Now I think I am done with diff, except one thing. And this is quite an incompatible change so I do not know how well it would work in practice. I am not even advocating this. It is more like me thinking aloud.

The diff-raw format we have been dealing with (sorry about '\t' vs ' ' gotcha again) is internally enhanced by diff-core. It first introduces entries for unmodified paths; '*' entries that has the same mode/sha1 in from->to pair are such entries, and that is what the change in the [PATCH 3/3] is about.

    *100644->100644 blob 233a250...->66818b4... file0
    *100755->100755 blob fc77389...->7b72d3d... file1
    +100644 blob 233a250... file2
    -100755 blob fc77389... file3
    *100644->100644 blob 233a250...->233a250... file4

Then diff-core internally extends the format to make things all look like this ('*' and '-' are gone and each record acquires the second path).

    100644->100644 233a250...->66818b4... file0 file0
    100755->100755 fc77389...->7b72d3d... file1 file1
    ______->100644 _______...->233a250... file2 file2
    100755->______ fc77389...->_______... file3 file3
    100644->100644 233a250...->233a250... file4 file4

Internally "______" above are represented with a separate flag (file_valid), and denotes the absense of either src or dst.

The diff-core is all about manipulating this type of list and changing one such list into a different list.

For example, rename-edit of file3 into file2 is detected by diffcore-rename module and these entries:

    ______->100644 _______...->233a250... file2 file2
    100755->______ fc77389...->_______... file3 file3

become:

    100755->100644 fc77389...->233a250... file3 file2

What the diffcore-pickaxe does can also be explained clearly with this model. It takes such a list and works as a "grep".

Once we start to think of it this way, it becomes quite tempting to change the diff-raw format to actually match the above concept. Meaning, (1) drop the operation letter +/-/* (inferrable by looking at the both sides of ->); (2) drop blob/tree (inferrable it from mode); (3) give two paths (usually they are the same paths); (4) and perhaps replace '->' with the same column separator. Like this:

    100644 100644 233a250... 66818b4... file0 file0
    100755 100755 fc77389... 7b72d3d... file1 file1
    ______ 100644 _______... 233a250... file2 file2
    100755 ______ fc77389... _______... file3 file3
    100644 100644 233a250... 233a250... file4 file4

Again, I am not even advocating this. It is more like me still thinking aloud.