Re: [PATCH 3/3] Diff overhaul, adding the other half...

From: Linus Torvalds <torvalds@osdl.org>
Date: May 21, 2005, 18:45 UTC
Message-ID: <Pine.LNX.4.58.0505211137250.2206@ppc970.osdl.org>
In-Reply-To: <Pine.LNX.4.58.0505211128570.2206@ppc970.osdl.org>

On Sat, 21 May 2005, Linus Torvalds wrote:

> 
> Which is a sensible default, and I note that you sent a separate email for 
> testing the extreme case. I'll try that out too, just for fun,

Hmm.. It's not working well. Not only does it take a lot of CPU time (do an fsck first to make sure you're not seekign the disk all over the place), but it "finds" lots of things like this:

	diff --git a/drivers/usb/misc/emi62_fw_m.h b/drivers/media/dvb/bt8xx/dst_ca.h
	similarity index 99%
	copy from drivers/usb/misc/emi62_fw_m.h
	copy to drivers/media/dvb/bt8xx/dst_ca.h
	--- a/drivers/usb/misc/emi62_fw_m.h
	+++ b/drivers/media/dvb/bt8xx/dst_ca.h
	@@ -1,8853 +1,58 @@
	 /*
	- * This file is generated from three different files, provided by Emagic.
	- */
	-/* generated Tue Jun 3 21:36:11 EEST 2003 */
	+       CA-driver for TwinHan DST Frontend/Card
 
which looks quite bogus (it they aren't similar at all, and the diff is 
huge).

I think that your similarity check has a tendency to do bad things if one of the files is huge: in this case we have

	torvalds@ppc970:~/v2.6/linux> wc -c drivers/media/dvb/bt8xx/dst_ca.h drivers/usb/misc/emi62_fw_m.h
	  1591 drivers/media/dvb/bt8xx/dst_ca.h
	795679 drivers/usb/misc/emi62_fw_m.h
	797270 total

and you consider them "similar", probably because it turns out that a delta that just removes everything is very small (it's just a "delete bytes x-y") so you compare that "small" delta to a "large total file" and you think it's an almost perfect match.

Now, for _renames_ that is actually half-way the right thing to do, but for copies, you should compare the size not to the _sum_ of the two files, but to just the size of the file that you generate.

But it's a fun example ;)

		Linus