{"thread":{"id":"28147","subject":"[PATCH 0/4] Support threshold in copy/rename detection","startedAt":"2011-08-18T01:04:15Z","lastAt":"2011-08-18T21:50:36Z","messageCount":7,"participants":["Vitor Antunes","Pete Wyckoff"],"isPatch":true,"patchVersion":1,"patchTotal":4},"messages":[{"id":"173705","messageId":"1313629459-22937-1-git-send-email-vitor.hda@gmail.com","threadId":"28147","inReplyTo":null,"subject":"[PATCH 0/4] Support threshold in copy/rename detection","fromName":"Vitor Antunes","fromEmail":"vitor.hda@gmail.com","sentAt":"2011-08-18T01:04:15Z","receivedAt":"2011-08-18T01:04:15Z","isPatch":true,"sender":{"key":"vitor.hda@gmail.com","avatar":"https://avatars.githubusercontent.com/u/606876?v=4"},"body":"Add possibility of configuring a threshold in the copy and rename detections.\nCopy and rename detection options were not documented and no test cases existed\nfor it. This is corrected in this set of patches.\n\nVitor Antunes (4):\n git-p4: Allow setting rename/copy detection threshold.\n git-p4: Add description of rename/copy detection options.\n git-p4: Add test case for rename detection.\n git-p4: Add test case for copy detection.\n\n contrib/fast-import/git-p4 | 18 ++++--\n contrib/fast-import/git-p4.txt | 25 ++++++++\n t/t9800-git-p4.sh | 121 ++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 159 insertions(+), 5 deletions(-)\n\n-- \n1.7.5.4\n"},{"id":"173706","messageId":"1313629459-22937-2-git-send-email-vitor.hda@gmail.com","threadId":"28147","inReplyTo":"1313629459-22937-1-git-send-email-vitor.hda@gmail.com","subject":"[PATCH 1/4] git-p4: Allow setting rename/copy detection threshold.","fromName":"Vitor Antunes","fromEmail":"vitor.hda@gmail.com","sentAt":"2011-08-18T01:04:16Z","receivedAt":"2011-08-18T01:04:16Z","isPatch":true,"sender":{"key":"vitor.hda@gmail.com","avatar":"https://avatars.githubusercontent.com/u/606876?v=4"},"body":"Copy and rename detection arguments (-C and -M) allow setting a threshold value\nfor the similarity ratio. If the similarity is below this threshold the rename\nor copy is ignored and the file is added as new.\nThis patch allows setting git-p4.detectRenames and git-p4.detectCopies options\nto an integer value to set the respective threshold.\n\nSigned-off-by: Vitor Antunes \n---\n contrib/fast-import/git-p4 | 18 +++++++++++++-----\n 1 files changed, 13 insertions(+), 5 deletions(-)\n\ndiff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4\nindex 6b9de9e..cf719be 100755\n--- a/contrib/fast-import/git-p4\n+++ b/contrib/fast-import/git-p4\n@@ -774,15 +774,23 @@ class P4Submit(Command, P4UserMap):\n \n if not self.detectRenames:\n # If not explicitly set check the config variable\n- self.detectRenames = gitConfig(\"git-p4.detectRenames\").lower() == \"true\"\n+ self.detectRenames = gitConfig(\"git-p4.detectRenames\")\n \n- if self.detectRenames:\n+ diffOpts = \"\"\n+ if self.detectRenames.lower() == \"true\":\n diffOpts = \"-M\"\n- else:\n- diffOpts = \"\"\n+ elif self.detectRenames != \"\":\n+ self.detectRenames = int(self.detectRenames)\n+ if self.detectRenames >= 0 and self.detectRenames <= 100:\n+ diffOpts = \"-M%d\" % self.detectRenames\n \n- if gitConfig(\"git-p4.detectCopies\").lower() == \"true\":\n+ detectCopies = gitConfig(\"git-p4.detectCopies\")\n+ if detectCopies.lower() == \"true\":\n diffOpts += \" -C\"\n+ elif detectCopies != \"\":\n+ detectCopies = int(detectCopies)\n+ if detectCopies >= 0 and detectCopies <= 100:\n+ diffOpts += \" -C%d\" % detectCopies\n \n if gitConfig(\"git-p4.detectCopiesHarder\").lower() == \"true\":\n diffOpts += \" --find-copies-harder\"\n-- \n1.7.5.4\n"},{"id":"173708","messageId":"1313629459-22937-3-git-send-email-vitor.hda@gmail.com","threadId":"28147","inReplyTo":"1313629459-22937-1-git-send-email-vitor.hda@gmail.com","subject":"[PATCH 2/4] git-p4: Add description of rename/copy detection options.","fromName":"Vitor Antunes","fromEmail":"vitor.hda@gmail.com","sentAt":"2011-08-18T01:04:17Z","receivedAt":"2011-08-18T01:04:17Z","isPatch":true,"sender":{"key":"vitor.hda@gmail.com","avatar":"https://avatars.githubusercontent.com/u/606876?v=4"},"body":"Signed-off-by: Vitor Antunes \n---\n contrib/fast-import/git-p4.txt | 25 +++++++++++++++++++++++++\n 1 files changed, 25 insertions(+), 0 deletions(-)\n\ndiff --git a/contrib/fast-import/git-p4.txt b/contrib/fast-import/git-p4.txt\nindex caa4bb3..2ffbccc 100644\n--- a/contrib/fast-import/git-p4.txt\n+++ b/contrib/fast-import/git-p4.txt\n@@ -232,6 +232,31 @@ git-p4.skipUserNameCheck\n When submitting, git-p4 checks that the git commits are authored by the current\n p4 user, and warns if they are not. This disables the check.\n \n+git-p4.detectRenames\n+\n+Detect renames when submitting changes to Perforce server. Will enable -M git\n+argument. Can be optionally set to a number representing the threshold\n+percentage value of the rename detection.\n+\n+ git config [--global] git-p4.detectRenames true\n+ git config [--global] git-p4.detectRenames 50\n+\n+git-p4.detectCopies\n+\n+Detect copies when submitting changes to Perforce server. Will enable -C git\n+argument. Can be optionally set to a number representing the threshold\n+percentage value of the copy detection.\n+\n+ git config [--global] git-p4.detectCopies true\n+ git config [--global] git-p4.detectCopies 80\n+\n+git-p4.detectCopiesHarder\n+\n+Detect copies even between files that did not change when submitting changes to\n+Perforce server. Will enable --find-copies-harder git argument.\n+\n+ git config [--global] git-p4.detectCopies true\n+\n Implementation Details...\n =========================\n \n-- \n1.7.5.4\n"},{"id":"173707","messageId":"1313629459-22937-4-git-send-email-vitor.hda@gmail.com","threadId":"28147","inReplyTo":"1313629459-22937-1-git-send-email-vitor.hda@gmail.com","subject":"[PATCH 3/4] git-p4: Add test case for rename detection.","fromName":"Vitor Antunes","fromEmail":"vitor.hda@gmail.com","sentAt":"2011-08-18T01:04:18Z","receivedAt":"2011-08-18T01:04:18Z","isPatch":true,"sender":{"key":"vitor.hda@gmail.com","avatar":"https://avatars.githubusercontent.com/u/606876?v=4"},"body":"Signed-off-by: Vitor Antunes \n---\n t/t9800-git-p4.sh | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++\n 1 files changed, 50 insertions(+), 0 deletions(-)\n\ndiff --git a/t/t9800-git-p4.sh b/t/t9800-git-p4.sh\nindex 97ec975..d01a1cb 100755\n--- a/t/t9800-git-p4.sh\n+++ b/t/t9800-git-p4.sh\n@@ -269,6 +269,56 @@ test_expect_success 'initial import time from top change time' '\n \ttest $p4time = $gittime\n '\n \n+# Rename a file and confirm that rename is not detected in P4.\n+# Rename the new file again with detectRenames option enabled and confirm that\n+# this is detected in P4.\n+# Rename the new file again adding an extra blank line, configure a big\n+# threshold in detectRenames and confirm that rename is not detected in P4.\n+# Rename the new file again adding another extra blank line, configure a small\n+# threshold in detectRenames and confirm that rename is not detected in P4.\n+test_expect_success 'detect renames' '\n+\tgit init \"$git\" &&\n+\tcd \"$git\" &&\n+\tcd \"$TRASH_DIRECTORY\" &&\n+\t\"$GITP4\" clone --dest=\"$git\" //depot@all &&\n+\tp4 files //depot/* &&\n+\tcd \"$git\" &&\n+\tgit mv file1 file4 &&\n+\tgit commit -a -m \"Rename file1 to file4\" &&\n+\tgit diff-tree -r -M HEAD &&\n+\tgit config git-p4.skipSubmitEditCheck true &&\n+\t\"$GITP4\" submit &&\n+\tp4 filelog //depot/file4 &&\n+\t! p4 filelog //depot/file4 | grep -q \"branch from //depot/file1\" &&\n+\tgit mv file4 file5 &&\n+\tgit commit -a -m \"Rename file4 to file5\" &&\n+\tgit diff-tree -r -M HEAD &&\n+\tgit config git-p4.detectRenames true &&\n+\t\"$GITP4\" submit &&\n+\tp4 filelog //depot/file5 &&\n+\tp4 filelog //depot/file5 | grep -q \"branch from //depot/file4\" &&\n+\tgit mv file5 file6 &&\n+\techo update >> file6 &&\n+\tgit add file6 &&\n+\tgit commit -a -m \"Rename file5 to file6 with changes\" &&\n+\tgit diff-tree -r -M HEAD &&\n+\tgit config git-p4.detectRenames 95 &&\n+\t\"$GITP4\" submit &&\n+\tp4 filelog //depot/file6 &&\n+\t! p4 filelog //depot/file6 | grep -q \"branch from //depot/file5\" &&\n+\tgit mv file6 file7 &&\n+\techo update >> file7 &&\n+\tgit add file7 &&\n+\tgit commit -a -m \"Rename file6 to file7 with changes\" &&\n+\tgit diff-tree -r -M HEAD &&\n+\tgit config git-p4.detectRenames 80 &&\n+\t\"$GITP4\" submit &&\n+\tp4 filelog //depot/file7 &&\n+\tp4 filelog //depot/file7 | grep -q \"branch from //depot/file6\" &&\n+\tcd \"$TRASH_DIRECTORY\" &&\n+\trm -rf \"$git\" && mkdir \"$git\"\n+'\n+\n test_expect_success 'shutdown' '\n \tpid=`pgrep -f p4d` &&\n \ttest -n \"$pid\" &&\n-- \n1.7.5.4\n"},{"id":"173709","messageId":"1313629459-22937-5-git-send-email-vitor.hda@gmail.com","threadId":"28147","inReplyTo":"1313629459-22937-1-git-send-email-vitor.hda@gmail.com","subject":"[PATCH 4/4] git-p4: Add test case for copy detection.","fromName":"Vitor Antunes","fromEmail":"vitor.hda@gmail.com","sentAt":"2011-08-18T01:04:19Z","receivedAt":"2011-08-18T01:04:19Z","isPatch":true,"sender":{"key":"vitor.hda@gmail.com","avatar":"https://avatars.githubusercontent.com/u/606876?v=4"},"body":"Signed-off-by: Vitor Antunes \n---\n t/t9800-git-p4.sh | 71 +++++++++++++++++++++++++++++++++++++++++++++++++++++\n 1 files changed, 71 insertions(+), 0 deletions(-)\n\ndiff --git a/t/t9800-git-p4.sh b/t/t9800-git-p4.sh\nindex d01a1cb..a4f3d66 100755\n--- a/t/t9800-git-p4.sh\n+++ b/t/t9800-git-p4.sh\n@@ -319,6 +319,77 @@ test_expect_success 'detect renames' '\n \trm -rf \"$git\" && mkdir \"$git\"\n '\n \n+# Copy a file and confirm that copy is not detected in P4.\n+# Copy a file with detectCopies option enabled and confirm that copy is not\n+# detected in P4.\n+# Modify and copy a file with detectCopies option enabled and confirm that copy\n+# is detected in P4.\n+# Copy a file with detectCopies and detectCopiesHarder options enabled and\n+# confirm that copy is detected in P4.\n+# Modify and copy a file, configure a big threshold in detectCopies and confirm\n+# that copy is not detected in P4.\n+# Modify and copy a file, configure a small threshold in detectCopies and\n+# confirm that copy is detected in P4.\n+test_expect_success 'detect copies' '\n+\tgit init \"$git\" &&\n+\tcd \"$git\" &&\n+\tcd \"$TRASH_DIRECTORY\" &&\n+\t\"$GITP4\" clone --dest=\"$git\" //depot@all &&\n+\tcd \"$git\" &&\n+\tcp file2 file8 &&\n+\tgit add file8 &&\n+\tgit commit -a -m \"Copy file2 to file8\" &&\n+\tgit diff-tree -r -C HEAD\n+\tgit config git-p4.skipSubmitEditCheck true &&\n+\t\"$GITP4\" submit &&\n+\tp4 filelog //depot/file8 &&\n+\t! p4 filelog //depot/file8 | grep -q \"branch from //depot/file\" &&\n+\tcp file2 file9 &&\n+\tgit add file9 &&\n+\tgit commit -a -m \"Copy file2 to file9\" &&\n+\tgit diff-tree -r -C HEAD\n+\tgit config git-p4.detectCopies true &&\n+\t\"$GITP4\" submit &&\n+\tp4 filelog //depot/file9 &&\n+\t! p4 filelog //depot/file9 | grep -q \"branch from //depot/file\" &&\n+\techo file2 >> file2 &&\n+\tcp file2 file10 &&\n+\tgit add file2 file10 &&\n+\tgit commit -a -m \"Modify and copy file2 to file10\" &&\n+\tgit diff-tree -r -C HEAD\n+\t\"$GITP4\" submit &&\n+\tp4 filelog //depot/file10 &&\n+\tp4 filelog //depot/file10 | grep -q \"branch from //depot/file\" &&\n+\tcp file2 file11 &&\n+\tgit add file11 &&\n+\tgit commit -a -m \"Copy file2 to file11\" &&\n+\tgit diff-tree -r -C --find-copies-harder HEAD\n+\tgit config git-p4.detectCopiesHarder true &&\n+\t\"$GITP4\" submit &&\n+\tp4 filelog //depot/file11 &&\n+\tp4 filelog //depot/file11 | grep -q \"branch from //depot/file\" &&\n+\tcp file2 file12 &&\n+\techo >> file12 &&\n+\tgit add file12 &&\n+\tgit commit -a -m \"Copy file2 to file12 with changes\" &&\n+\tgit diff-tree -r -C --find-copies-harder HEAD\n+\tgit config git-p4.detectCopies 98 &&\n+\t\"$GITP4\" submit &&\n+\tp4 filelog //depot/file12 &&\n+\t! p4 filelog //depot/file12 | grep -q \"branch from //depot/file\" &&\n+\tcp file2 file13 &&\n+\techo >> file13 &&\n+\tgit add file13 &&\n+\tgit commit -a -m \"Copy file2 to file13 with changes\" &&\n+\tgit diff-tree -r -C --find-copies-harder HEAD\n+\tgit config git-p4.detectCopies 80 &&\n+\t\"$GITP4\" submit &&\n+\tp4 filelog //depot/file13 &&\n+\tp4 filelog //depot/file13 | grep -q \"branch from //depot/file\" &&\n+\tcd \"$TRASH_DIRECTORY\" &&\n+\trm -rf \"$git\" && mkdir \"$git\"\n+'\n+\n test_expect_success 'shutdown' '\n \tpid=`pgrep -f p4d` &&\n \ttest -n \"$pid\" &&\n-- \n1.7.5.4\n"},{"id":"173762","messageId":"20110818151847.GA2043@padd.com","threadId":"28147","inReplyTo":"1313629459-22937-2-git-send-email-vitor.hda@gmail.com","subject":"Re: [PATCH 1/4] git-p4: Allow setting rename/copy detection threshold.","fromName":"Pete Wyckoff","fromEmail":"pw@padd.com","sentAt":"2011-08-18T15:18:47Z","receivedAt":"2011-08-18T15:18:47Z","isPatch":true,"sender":{"key":"pw@padd.com","avatar":null},"body":"vitor.hda@gmail.com wrote on Thu, 18 Aug 2011 02:04 +0100:\n> Copy and rename detection arguments (-C and -M) allow setting a threshold value\n> for the similarity ratio. If the similarity is below this threshold the rename\n> or copy is ignored and the file is added as new.\n> This patch allows setting git-p4.detectRenames and git-p4.detectCopies options\n> to an integer value to set the respective threshold.\n[..]\n> - if self.detectRenames:\n> + diffOpts = \"\"\n> + if self.detectRenames.lower() == \"true\":\n> diffOpts = \"-M\"\n> - else:\n> - diffOpts = \"\"\n> + elif self.detectRenames != \"\":\n> + self.detectRenames = int(self.detectRenames)\n> + if self.detectRenames >= 0 and self.detectRenames <= 100:\n> + diffOpts = \"-M%d\" % self.detectRenames\n> \n\nThis is a good idea. But I notice that -M, -C, -B can take\na diverse set of option forms, including floating point and\noptional \"%\". \"-B\" can take two args, separated by a \"/\".\n\nSee diff_scoreopt_parse(). It would be weird for git-p4 to limit\nthat set to integers like you do here.\n\nWould you be willing to change it to pass the string blindly?\nJust filter out the special values, and pass everything else\nunchanged.\n\n \"true\" -> \"-M\"\n \"false\" -> \"\"\n \"\" -> \"\"\n \".*\" -> \"\\1\"\n\nDunno if you care about --break-rewrites. Just pointing to it\nas a similar option to -M and -C in case you missed it.\n\n\t\t-- Pete\n"},{"id":"173787","messageId":"CAOpHH-VVniVZCG3KayLo8-C95Mhdj4e4gQTSyeMp2orNkOD9bA@mail.gmail.com","threadId":"28147","inReplyTo":"20110818151847.GA2043@padd.com","subject":"Re: [PATCH 1/4] git-p4: Allow setting rename/copy detection threshold.","fromName":"Vitor Antunes","fromEmail":"vitor.hda@gmail.com","sentAt":"2011-08-18T21:50:36Z","receivedAt":"2011-08-18T21:50:36Z","isPatch":true,"sender":{"key":"vitor.hda@gmail.com","avatar":"https://avatars.githubusercontent.com/u/606876?v=4"},"body":"On Thu, Aug 18, 2011 at 4:18 PM, Pete Wyckoff wrote:\n> This is a good idea.  But I notice that -M, -C, -B can take\n> a diverse set of option forms, including floating point and\n> optional \"%\".  \"-B\" can take two args, separated by a \"/\".\n>\n> See diff_scoreopt_parse().  It would be weird for git-p4 to limit\n> that set to integers like you do here.\n>\n> Would you be willing to change it to pass the string blindly?\n> Just filter out the special values, and pass everything else\n> unchanged.\n>\n>    \"true\" -> \"-M\"\n>    \"false\" -> \"\"\n>    \"\" -> \"\"\n>    \".*\" -> \"\\1\"\n\nThis makes sense. I did notice that -M and -C supported, at least, a\n'%' char at the end. But I decided to ignore it. I think your is idea\nis not only good, but also a simpler solution. I'll send a new set of\npatches shortly with this implemented.\n\n> Dunno if you care about --break-rewrites.  Just pointing to it\n> as a similar option to -M and -C in case you missed it.\n\nI was completely unaware of this option, so I don't feel comfortable\nin adding it as a configuration option for now. Will need to\nunderstand it first ;)\n\n-- \nVitor Antunes\n"}]}