{"thread":{"id":"30952","subject":"[PATCH 0/2] git p4: use \"move\" command for renames","startedAt":"2012-07-04T13:40:18Z","lastAt":"2012-07-09T10:56:29Z","messageCount":5,"participants":["Pete Wyckoff","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"194620","messageId":"1341409220-27954-1-git-send-email-pw@padd.com","threadId":"30952","inReplyTo":null,"subject":"[PATCH 0/2] git p4: use \"move\" command for renames","fromName":"Pete Wyckoff","fromEmail":"pw@padd.com","sentAt":"2012-07-04T13:40:18Z","receivedAt":"2012-07-04T13:40:18Z","isPatch":true,"sender":{"key":"pw@padd.com","avatar":null},"body":"Recent p4 supports a \"move\" command that records explicitly that\na file was moved from one place to another.  It can be changed a bit\nduring the move, too.  Use this feature, if it exists, when renames\nare detected.\n\nGary sent these patches months ago, and I've been sitting on them\nfar too long.  Was hoping to move some other work out first, but it\nwas not ready.\n\nThese commits are on origin/next, as they depend on changes\nin pw/git-p4-tests that was merged into next on Jul 3.  They\ndo not conflict with the other series in flight, \"notice Jobs: ...\".\n\nGary Gibbons (2):\n  git p4: refactor diffOpts calculation\n  git p4: add support for 'p4 move' in P4Submit\n\n Documentation/git-p4.txt | 10 +++---\n git-p4.py                | 86 ++++++++++++++++++++++++++++++++----------------\n t/t9814-git-p4-rename.sh | 16 ++++-----\n 3 files changed, 72 insertions(+), 40 deletions(-)\n\n-- \n1.7.11.1.125.g4a65fea\n"},{"id":"194621","messageId":"1341409220-27954-2-git-send-email-pw@padd.com","threadId":"30952","inReplyTo":"1341409220-27954-1-git-send-email-pw@padd.com","subject":"[PATCH 1/2] git p4: refactor diffOpts calculation","fromName":"Pete Wyckoff","fromEmail":"pw@padd.com","sentAt":"2012-07-04T13:40:19Z","receivedAt":"2012-07-04T13:40:19Z","isPatch":true,"sender":{"key":"pw@padd.com","avatar":null},"body":"From: Gary Gibbons <ggibbons@perforce.com>\n\nP4Submit.applyCommit()\n\nTo avoid recalculating the same diffOpts for each commit, move it\nout of applyCommit() and into the top-level run().  Also fix a bug\nin that code which interpreted the value of detectRenames as a\nstring rather than as a boolean.\n\n[pw: fix documentation, rearrange code a bit]\n\nSigned-off-by: Gary Gibbons <ggibbons@perforce.com>\nSigned-off-by: Pete Wyckoff <pw@padd.com>\n---\n Documentation/git-p4.txt | 10 ++++++----\n git-p4.py                | 52 +++++++++++++++++++++++++++++-------------------\n 2 files changed, 38 insertions(+), 24 deletions(-)\n\ndiff --git a/Documentation/git-p4.txt b/Documentation/git-p4.txt\nindex fe1f49b..8228f33 100644\n--- a/Documentation/git-p4.txt\n+++ b/Documentation/git-p4.txt\n@@ -255,7 +255,7 @@ These options can be used to modify 'git p4 submit' behavior.\n \tp4.  By default, this is the most recent p4 commit reachable\n \tfrom 'HEAD'.\n \n--M[<n>]::\n+-M::\n \tDetect renames.  See linkgit:git-diff[1].  Renames will be\n \trepresented in p4 using explicit 'move' operations.  There\n \tis no corresponding option to detect copies, but there are\n@@ -465,13 +465,15 @@ git-p4.useClientSpec::\n Submit variables\n ~~~~~~~~~~~~~~~~\n git-p4.detectRenames::\n-\tDetect renames.  See linkgit:git-diff[1].\n+\tDetect renames.  See linkgit:git-diff[1].  This can be true,\n+\tfalse, or a score as expected by 'git diff -M'.\n \n git-p4.detectCopies::\n-\tDetect copies.  See linkgit:git-diff[1].\n+\tDetect copies.  See linkgit:git-diff[1].  This can be true,\n+\tfalse, or a score as expected by 'git diff -C'.\n \n git-p4.detectCopiesHarder::\n-\tDetect copies harder.  See linkgit:git-diff[1].\n+\tDetect copies harder.  See linkgit:git-diff[1].  A boolean.\n \n git-p4.preserveUser::\n \tOn submit, re-author changes to reflect the git author,\ndiff --git a/git-p4.py b/git-p4.py\nindex f895a24..5fe509f 100755\n--- a/git-p4.py\n+++ b/git-p4.py\n@@ -1046,27 +1046,8 @@ class P4Submit(Command, P4UserMap):\n \n         (p4User, gitEmail) = self.p4UserForCommit(id)\n \n-        if not self.detectRenames:\n-            # If not explicitly set check the config variable\n-            self.detectRenames = gitConfig(\"git-p4.detectRenames\")\n-\n-        if self.detectRenames.lower() == \"false\" or self.detectRenames == \"\":\n-            diffOpts = \"\"\n-        elif self.detectRenames.lower() == \"true\":\n-            diffOpts = \"-M\"\n-        else:\n-            diffOpts = \"-M%s\" % self.detectRenames\n-\n-        detectCopies = gitConfig(\"git-p4.detectCopies\")\n-        if detectCopies.lower() == \"true\":\n-            diffOpts += \" -C\"\n-        elif detectCopies != \"\" and detectCopies.lower() != \"false\":\n-            diffOpts += \" -C%s\" % detectCopies\n \n-        if gitConfig(\"git-p4.detectCopiesHarder\", \"--bool\") == \"true\":\n-            diffOpts += \" --find-copies-harder\"\n-\n-        diff = read_pipe_lines(\"git diff-tree -r %s \\\"%s^\\\" \\\"%s\\\"\" % (diffOpts, id, id))\n+        diff = read_pipe_lines(\"git diff-tree -r %s \\\"%s^\\\" \\\"%s\\\"\" % (self.diffOpts, id, id))\n         filesToAdd = set()\n         filesToDelete = set()\n         editedFiles = set()\n@@ -1433,6 +1414,37 @@ class P4Submit(Command, P4UserMap):\n         if self.preserveUser:\n             self.checkValidP4Users(commits)\n \n+        #\n+        # Build up a set of options to be passed to diff when\n+        # submitting each commit to p4.\n+        #\n+        if self.detectRenames:\n+            # command-line -M arg\n+            self.diffOpts = \"-M\"\n+        else:\n+            # If not explicitly set check the config variable\n+            detectRenames = gitConfig(\"git-p4.detectRenames\")\n+\n+            if detectRenames.lower() == \"false\" or detectRenames == \"\":\n+                self.diffOpts = \"\"\n+            elif detectRenames.lower() == \"true\":\n+                self.diffOpts = \"-M\"\n+            else:\n+                self.diffOpts = \"-M%s\" % detectRenames\n+\n+        # no command-line arg for -C or --find-copies-harder, just\n+        # config variables\n+        detectCopies = gitConfig(\"git-p4.detectCopies\")\n+        if detectCopies.lower() == \"false\" or detectCopies == \"\":\n+            pass\n+        elif detectCopies.lower() == \"true\":\n+            self.diffOpts += \" -C\"\n+        else:\n+            self.diffOpts += \" -C%s\" % detectCopies\n+\n+        if gitConfig(\"git-p4.detectCopiesHarder\", \"--bool\") == \"true\":\n+            self.diffOpts += \" --find-copies-harder\"\n+\n         while len(commits) > 0:\n             commit = commits[0]\n             commits = commits[1:]\n-- \n1.7.11.1.125.g4a65fea\n"},{"id":"194622","messageId":"1341409220-27954-3-git-send-email-pw@padd.com","threadId":"30952","inReplyTo":"1341409220-27954-1-git-send-email-pw@padd.com","subject":"[PATCH 2/2] git p4: add support for 'p4 move' in P4Submit","fromName":"Pete Wyckoff","fromEmail":"pw@padd.com","sentAt":"2012-07-04T13:40:20Z","receivedAt":"2012-07-04T13:40:20Z","isPatch":true,"sender":{"key":"pw@padd.com","avatar":null},"body":"From: Gary Gibbons <ggibbons@perforce.com>\n\nFor -M option (detectRenames) in P4Submit, use 'p4 move' rather\nthan 'p4 integrate'.  Check Perforce server for exisitence of\n'p4 move' and use it if present, otherwise revert to 'p4 integrate'.\n\n[pw: wildcard-encode src/dest, add/update tests, tweak code]\n\nSigned-off-by: Gary Gibbons <ggibbons@perforce.com>\nSigned-off-by: Pete Wyckoff <pw@padd.com>\n---\n git-p4.py                | 34 ++++++++++++++++++++++++++--------\n t/t9814-git-p4-rename.sh | 16 ++++++++--------\n 2 files changed, 34 insertions(+), 16 deletions(-)\n\ndiff --git a/git-p4.py b/git-p4.py\nindex 5fe509f..b79e6f0 100755\n--- a/git-p4.py\n+++ b/git-p4.py\n@@ -120,6 +120,15 @@ def p4_read_pipe_lines(c):\n     real_cmd = p4_build_cmd(c)\n     return read_pipe_lines(real_cmd)\n \n+def p4_has_command(cmd):\n+    \"\"\"Ask p4 for help on this command.  If it returns an error, the\n+       command does not exist in this version of p4.\"\"\"\n+    real_cmd = p4_build_cmd([\"help\", cmd])\n+    p = subprocess.Popen(real_cmd, stdout=subprocess.PIPE,\n+                                   stderr=subprocess.PIPE)\n+    p.communicate()\n+    return p.returncode == 0\n+\n def system(cmd):\n     expand = isinstance(cmd,basestring)\n     if verbose:\n@@ -157,6 +166,9 @@ def p4_revert(f):\n def p4_reopen(type, f):\n     p4_system([\"reopen\", \"-t\", type, wildcard_encode(f)])\n \n+def p4_move(src, dest):\n+    p4_system([\"move\", \"-k\", wildcard_encode(src), wildcard_encode(dest)])\n+\n #\n # Canonicalize the p4 type and return a tuple of the\n # base type, plus any modifiers.  See \"p4 help filetypes\"\n@@ -850,6 +862,7 @@ class P4Submit(Command, P4UserMap):\n         self.preserveUser = gitConfig(\"git-p4.preserveUser\").lower() == \"true\"\n         self.isWindows = (platform.system() == \"Windows\")\n         self.exportLabels = False\n+        self.p4HasMoveCommand = p4_has_command(\"move\") \n \n     def check(self):\n         if len(p4CmdList(\"opened ...\")) > 0:\n@@ -1046,7 +1059,6 @@ class P4Submit(Command, P4UserMap):\n \n         (p4User, gitEmail) = self.p4UserForCommit(id)\n \n-\n         diff = read_pipe_lines(\"git diff-tree -r %s \\\"%s^\\\" \\\"%s\\\"\" % (self.diffOpts, id, id))\n         filesToAdd = set()\n         filesToDelete = set()\n@@ -1087,17 +1099,23 @@ class P4Submit(Command, P4UserMap):\n                 editedFiles.add(dest)\n             elif modifier == \"R\":\n                 src, dest = diff['src'], diff['dst']\n-                p4_integrate(src, dest)\n-                if diff['src_sha1'] != diff['dst_sha1']:\n-                    p4_edit(dest)\n+                if self.p4HasMoveCommand:\n+                    p4_edit(src)        # src must be open before move\n+                    p4_move(src, dest)  # opens for (move/delete, move/add)\n                 else:\n-                    pureRenameCopy.add(dest)\n+                    p4_integrate(src, dest)\n+                    if diff['src_sha1'] != diff['dst_sha1']:\n+                        p4_edit(dest)\n+                    else:\n+                        pureRenameCopy.add(dest)\n                 if isModeExecChanged(diff['src_mode'], diff['dst_mode']):\n-                    p4_edit(dest)\n+                    if not self.p4HasMoveCommand:\n+                        p4_edit(dest)   # with move: already open, writable\n                     filesToChangeExecBit[dest] = diff['dst_mode']\n-                os.unlink(dest)\n+                if not self.p4HasMoveCommand:\n+                    os.unlink(dest)\n+                    filesToDelete.add(src)\n                 editedFiles.add(dest)\n-                filesToDelete.add(src)\n             else:\n                 die(\"unknown modifier %s for %s\" % (modifier, path))\n \ndiff --git a/t/t9814-git-p4-rename.sh b/t/t9814-git-p4-rename.sh\nindex 84fffb3..8be74b6 100755\n--- a/t/t9814-git-p4-rename.sh\n+++ b/t/t9814-git-p4-rename.sh\n@@ -77,16 +77,16 @@ test_expect_success 'detect renames' '\n \t\tgit commit -a -m \"Rename file1 to file4\" &&\n \t\tgit diff-tree -r -M HEAD &&\n \t\tgit p4 submit &&\n-\t\tp4 filelog //depot/file4 &&\n-\t\tp4 filelog //depot/file4 | test_must_fail grep -q \"branch from\" &&\n+\t\tp4 filelog //depot/file4 | tee filelog &&\n+\t\t! grep -q \" from //depot\" filelog &&\n \n \t\tgit mv file4 file5 &&\n \t\tgit commit -a -m \"Rename file4 to file5\" &&\n \t\tgit diff-tree -r -M HEAD &&\n \t\tgit config git-p4.detectRenames true &&\n \t\tgit p4 submit &&\n-\t\tp4 filelog //depot/file5 &&\n-\t\tp4 filelog //depot/file5 | grep -q \"branch from //depot/file4\" &&\n+\t\tp4 filelog //depot/file5 | tee filelog &&\n+\t\tgrep -q \" from //depot/file4\" filelog &&\n \n \t\tgit mv file5 file6 &&\n \t\techo update >>file6 &&\n@@ -97,8 +97,8 @@ test_expect_success 'detect renames' '\n \t\ttest -n \"$level\" && test \"$level\" -gt 0 && test \"$level\" -lt 98 &&\n \t\tgit config git-p4.detectRenames $(($level + 2)) &&\n \t\tgit p4 submit &&\n-\t\tp4 filelog //depot/file6 &&\n-\t\tp4 filelog //depot/file6 | test_must_fail grep -q \"branch from\" &&\n+\t\tp4 filelog //depot/file6 | tee filelog &&\n+\t\t! grep -q \" from //depot\" filelog &&\n \n \t\tgit mv file6 file7 &&\n \t\techo update >>file7 &&\n@@ -109,8 +109,8 @@ test_expect_success 'detect renames' '\n \t\ttest -n \"$level\" && test \"$level\" -gt 2 && test \"$level\" -lt 100 &&\n \t\tgit config git-p4.detectRenames $(($level - 2)) &&\n \t\tgit p4 submit &&\n-\t\tp4 filelog //depot/file7 &&\n-\t\tp4 filelog //depot/file7 | grep -q \"branch from //depot/file6\"\n+\t\tp4 filelog //depot/file7 | tee filelog &&\n+\t\tgrep -q \" from //depot/file6\" filelog\n \t)\n '\n \n-- \n1.7.11.1.125.g4a65fea\n"},{"id":"194693","messageId":"7v7guhpfmn.fsf@alter.siamese.dyndns.org","threadId":"30952","inReplyTo":"1341409220-27954-3-git-send-email-pw@padd.com","subject":"Re: [PATCH 2/2] git p4: add support for 'p4 move' in P4Submit","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-07-06T06:28:48Z","receivedAt":"2012-07-06T06:28:48Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Pete Wyckoff <pw@padd.com> writes:\n\n> diff --git a/t/t9814-git-p4-rename.sh b/t/t9814-git-p4-rename.sh\n> index 84fffb3..8be74b6 100755\n> --- a/t/t9814-git-p4-rename.sh\n> +++ b/t/t9814-git-p4-rename.sh\n> @@ -77,16 +77,16 @@ test_expect_success 'detect renames' '\n>  \t\tgit commit -a -m \"Rename file1 to file4\" &&\n>  \t\tgit diff-tree -r -M HEAD &&\n>  \t\tgit p4 submit &&\n> -\t\tp4 filelog //depot/file4 &&\n> -\t\tp4 filelog //depot/file4 | test_must_fail grep -q \"branch from\" &&\n> +\t\tp4 filelog //depot/file4 | tee filelog &&\n> +\t\t! grep -q \" from //depot\" filelog &&\n\nI am not a huge fan of using \"tee\" in our test scripts, especially\nas it means piping output of another command whose output (and\npresumably the behaviour) we care about, hiding its exit status.\n\nFixing the incorrect use of piping to \"test_must_fail grep\" is a\ngood change, but is there anything wrong to do the above like this?\n\n\tp4 filelog //depot/file4 >filelog &&\n\t! grep -q \" from //depot\" filelog &&\n"},{"id":"194805","messageId":"20120709105629.GA23746@padd.com","threadId":"30952","inReplyTo":"7v7guhpfmn.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 2/2] git p4: add support for 'p4 move' in P4Submit","fromName":"Pete Wyckoff","fromEmail":"pw@padd.com","sentAt":"2012-07-09T10:56:29Z","receivedAt":"2012-07-09T10:56:29Z","isPatch":true,"sender":{"key":"pw@padd.com","avatar":null},"body":"gitster@pobox.com wrote on Thu, 05 Jul 2012 23:28 -0700:\n> Pete Wyckoff <pw@padd.com> writes:\n> \n> > diff --git a/t/t9814-git-p4-rename.sh b/t/t9814-git-p4-rename.sh\n> > index 84fffb3..8be74b6 100755\n> > --- a/t/t9814-git-p4-rename.sh\n> > +++ b/t/t9814-git-p4-rename.sh\n> > @@ -77,16 +77,16 @@ test_expect_success 'detect renames' '\n> >  \t\tgit commit -a -m \"Rename file1 to file4\" &&\n> >  \t\tgit diff-tree -r -M HEAD &&\n> >  \t\tgit p4 submit &&\n> > -\t\tp4 filelog //depot/file4 &&\n> > -\t\tp4 filelog //depot/file4 | test_must_fail grep -q \"branch from\" &&\n> > +\t\tp4 filelog //depot/file4 | tee filelog &&\n> > +\t\t! grep -q \" from //depot\" filelog &&\n> \n> I am not a huge fan of using \"tee\" in our test scripts, especially\n> as it means piping output of another command whose output (and\n> presumably the behaviour) we care about, hiding its exit status.\n> \n> Fixing the incorrect use of piping to \"test_must_fail grep\" is a\n> good change, but is there anything wrong to do the above like this?\n> \n> \tp4 filelog //depot/file4 >filelog &&\n> \t! grep -q \" from //depot\" filelog &&\n\nI'd started growing fond of \"tee\" as it shows all the\noutput, and isolates the grep as a separate step.  Much\neasier to see the bad output when a test fails.\n\nI'll switch around to your approach, adding a \"cat filelog\" line\nfor interesting cases.\n\n\t\t-- Pete\n"}]}