{"thread":{"id":"29447","subject":"[PATCH v3 0/3] git-p4: Search for parent commit on branch creation","startedAt":"2012-01-25T23:48:21Z","lastAt":"2012-01-26T04:21:49Z","messageCount":5,"participants":["Vitor Antunes","Pete Wyckoff"],"isPatch":true,"patchVersion":3,"patchTotal":3},"messages":[{"id":"183103","messageId":"1327535304-11332-1-git-send-email-vitor.hda@gmail.com","threadId":"29447","inReplyTo":null,"subject":"[PATCH v3 0/3] git-p4: Search for parent commit on branch creation","fromName":"Vitor Antunes","fromEmail":"vitor.hda@gmail.com","sentAt":"2012-01-25T23:48:21Z","receivedAt":"2012-01-25T23:48:21Z","isPatch":true,"sender":{"key":"vitor.hda@gmail.com","avatar":"https://avatars.githubusercontent.com/u/606876?v=4"},"body":"I think this will, hopefully, be the final version of this series of\npatches. This version includes the following changes since v2:\n\n - Move search algorithm into its own function.\n - Use lists instead of strings on shell commands.\n - Some small (almost cosmetic) updates to test cases.\n\nPete Wyckoff (1):\n  git-p4: Change p4 command invocation\n\nVitor Antunes (2):\n  git-p4: Search for parent commit on branch creation\n  git-p4: Add test case for complex branch import\n\n contrib/fast-import/git-p4 |   48 +++++++++++++++++++++-\n t/t9801-git-p4-branch.sh   |   94 ++++++++++++++++++++++++++++++++++++++++---\n 2 files changed, 133 insertions(+), 9 deletions(-)\n\n-- \n1.7.8.3\n"},{"id":"183105","messageId":"1327535304-11332-2-git-send-email-vitor.hda@gmail.com","threadId":"29447","inReplyTo":"1327535304-11332-1-git-send-email-vitor.hda@gmail.com","subject":"[PATCH v3 1/3] git-p4: Search for parent commit on branch creation","fromName":"Vitor Antunes","fromEmail":"vitor.hda@gmail.com","sentAt":"2012-01-25T23:48:22Z","receivedAt":"2012-01-25T23:48:22Z","isPatch":true,"sender":{"key":"vitor.hda@gmail.com","avatar":"https://avatars.githubusercontent.com/u/606876?v=4"},"body":"To find out which is its parent the commit of the new branch is compared\nsequentially to each blob of the parent branch from the newest to the\noldest. The first blob which results in a zero diff is considered the\nparent commit. If none is found, then the commit is applied to the top\nof the parent branch.\n\nA fast-import \"checkpoint\" call is required because diff-tree is only\nable to work with blobs on disk. But most of these commits will not be\npart of the final imported tree, making fast-import fail. To avoid this,\nthe temporary branches are tracked and then removed at the end of the\nimport process.\n\nSigned-off-by: Vitor Antunes <vitor.hda@gmail.com>\n---\n contrib/fast-import/git-p4 |   46 +++++++++++++++++++++++++++++++++++++++++++-\n 1 files changed, 45 insertions(+), 1 deletions(-)\n\ndiff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4\nindex 3e1aa27..584cc41 100755\n--- a/contrib/fast-import/git-p4\n+++ b/contrib/fast-import/git-p4\n@@ -1429,6 +1429,8 @@ class P4Sync(Command, P4UserMap):\n         self.cloneExclude = []\n         self.useClientSpec = False\n         self.clientSpecDirs = None\n+        self.tempBranches = []\n+        self.tempBranchLocation = \"git-p4-tmp\"\n \n         if gitConfig(\"git-p4.syncFromOrigin\") == \"false\":\n             self.syncWithOrigin = False\n@@ -1450,6 +1452,14 @@ class P4Sync(Command, P4UserMap):\n                    .replace(\"%25\", \"%\")\n         return path\n \n+    # Force a checkpoint in fast-import and wait for it to finish\n+    def checkpoint(self):\n+        self.gitStream.write(\"checkpoint\\n\\n\")\n+        self.gitStream.write(\"progress checkpoint\\n\\n\")\n+        out = self.gitOutput.readline()\n+        if self.verbose:\n+            print \"checkpoint finished: \" + out\n+\n     def extractFilesFromCommit(self, commit):\n         self.cloneExclude = [re.sub(r\"\\.\\.\\.$\", \"\", path)\n                              for path in self.cloneExclude]\n@@ -1948,6 +1958,20 @@ class P4Sync(Command, P4UserMap):\n         self.importChanges(changes)\n         return True\n \n+    def searchParent(self, parent, branch, target):\n+        parentFound = False\n+        for blob in read_pipe_lines([\"git\", \"rev-list\", \"--reverse\", \"--no-merges\", parent]):\n+            blob = blob.strip()\n+            if len(read_pipe([\"git\", \"diff-tree\", blob, target])) == 0:\n+                parentFound = True\n+                if self.verbose:\n+                    print \"Found parent of %s in commit %s\" % (branch, blob)\n+                break\n+        if parentFound:\n+            return blob\n+        else:\n+            return None\n+\n     def importChanges(self, changes):\n         cnt = 1\n         for change in changes:\n@@ -2004,7 +2028,21 @@ class P4Sync(Command, P4UserMap):\n                             parent = self.initialParents[branch]\n                             del self.initialParents[branch]\n \n-                        self.commit(description, filesForCommit, branch, [branchPrefix], parent)\n+                        blob = None\n+                        if len(parent) > 0:\n+                            tempBranch = os.path.join(self.tempBranchLocation, \"%d\" % (change))\n+                            if self.verbose:\n+                                print \"Creating temporary branch: \" + tempBranch\n+                            self.commit(description, filesForCommit, tempBranch, [branchPrefix])\n+                            self.tempBranches.append(tempBranch)\n+                            self.checkpoint()\n+                            blob = self.searchParent(parent, branch, tempBranch)\n+                        if blob:\n+                            self.commit(description, filesForCommit, branch, [branchPrefix], blob)\n+                        else:\n+                            if self.verbose:\n+                                print \"Parent of %s not found. Committing into head of %s\" % (branch, parent)\n+                            self.commit(description, filesForCommit, branch, [branchPrefix], parent)\n                 else:\n                     files = self.extractFilesFromCommit(description)\n                     self.commit(description, files, self.branch, self.depotPaths,\n@@ -2339,6 +2377,12 @@ class P4Sync(Command, P4UserMap):\n         self.gitOutput.close()\n         self.gitError.close()\n \n+        # Cleanup temporary branches created during import\n+        if self.tempBranches != []:\n+            for branch in self.tempBranches:\n+                read_pipe(\"git update-ref -d %s\" % branch)\n+            os.rmdir(os.path.join(os.environ.get(\"GIT_DIR\", \".git\"), self.tempBranchLocation))\n+\n         return True\n \n class P4Rebase(Command):\n-- \n1.7.8.3\n"},{"id":"183106","messageId":"1327535304-11332-3-git-send-email-vitor.hda@gmail.com","threadId":"29447","inReplyTo":"1327535304-11332-1-git-send-email-vitor.hda@gmail.com","subject":"[PATCH v3 2/3] git-p4: Add test case for complex branch import","fromName":"Vitor Antunes","fromEmail":"vitor.hda@gmail.com","sentAt":"2012-01-25T23:48:23Z","receivedAt":"2012-01-25T23:48:23Z","isPatch":true,"sender":{"key":"vitor.hda@gmail.com","avatar":"https://avatars.githubusercontent.com/u/606876?v=4"},"body":"Check if branches created from old changelists are correctly imported.\nAlso included some updates to simple branch test so that both are\ncoherent in respect to each other.\n\nSigned-off-by: Vitor Antunes <vitor.hda@gmail.com>\n---\n t/t9801-git-p4-branch.sh |   94 ++++++++++++++++++++++++++++++++++++++++++---\n 1 files changed, 87 insertions(+), 7 deletions(-)\n\ndiff --git a/t/t9801-git-p4-branch.sh b/t/t9801-git-p4-branch.sh\nindex a25f18d..6ff713b 100755\n--- a/t/t9801-git-p4-branch.sh\n+++ b/t/t9801-git-p4-branch.sh\n@@ -172,9 +172,9 @@ test_expect_success 'add simple p4 branches' '\n \t\techo file1 >file1 &&\n \t\techo file2 >file2 &&\n \t\tp4 add file1 file2 &&\n-\t\tp4 submit -d \"branch1\" &&\n+\t\tp4 submit -d \"Create branch1\" &&\n \t\tp4 integrate //depot/branch1/... //depot/branch2/... &&\n-\t\tp4 submit -d \"branch2\" &&\n+\t\tp4 submit -d \"Integrate branch2 from branch1\" &&\n \t\techo file3 >file3 &&\n \t\tp4 add file3 &&\n \t\tp4 submit -d \"add file3 in branch1\" &&\n@@ -182,7 +182,7 @@ test_expect_success 'add simple p4 branches' '\n \t\techo update >>file2 &&\n \t\tp4 submit -d \"update file2 in branch1\" &&\n \t\tp4 integrate //depot/branch1/... //depot/branch3/... &&\n-\t\tp4 submit -d \"branch3\"\n+\t\tp4 submit -d \"Integrate branch3 from branch1\"\n \t)\n '\n \n@@ -203,17 +203,17 @@ test_expect_success 'git-p4 clone simple branches' '\n \t\ttest -f file1 &&\n \t\ttest -f file2 &&\n \t\ttest -f file3 &&\n-\t\tgrep -q update file2 &&\n+\t\tgrep update file2 &&\n \t\tgit reset --hard p4/depot/branch2 &&\n \t\ttest -f file1 &&\n \t\ttest -f file2 &&\n \t\ttest ! -f file3 &&\n-\t\ttest_must_fail grep -q update file2 &&\n+\t\ttest_must_fail grep update file2 &&\n \t\tgit reset --hard p4/depot/branch3 &&\n \t\ttest -f file1 &&\n \t\ttest -f file2 &&\n \t\ttest -f file3 &&\n-\t\tgrep -q update file2 &&\n+\t\tgrep update file2 &&\n \t\tcd \"$cli\" &&\n \t\tcd branch1 &&\n \t\tp4 edit file2 &&\n@@ -222,7 +222,87 @@ test_expect_success 'git-p4 clone simple branches' '\n \t\tcd \"$git\" &&\n \t\tgit reset --hard p4/depot/branch1 &&\n \t\t\"$GITP4\" rebase &&\n-\t\tgrep -q file2_ file2\n+\t\tgrep file2_ file2\n+\t)\n+'\n+\n+# Create a complex branch structure in P4 depot to check if they are correctly\n+# cloned. The branches are created from older changelists to check if git-p4 is\n+# able to correctly detect them.\n+# The final expected structure is:\n+# `branch1\n+# | `- file1\n+# | `- file2 (updated)\n+# | `- file3\n+# `branch2\n+# | `- file1\n+# | `- file2\n+# `branch3\n+# | `- file1\n+# | `- file2 (updated)\n+# | `- file3\n+# `branch4\n+# | `- file1\n+# | `- file2\n+# `branch5\n+#   `- file1\n+#   `- file2\n+#   `- file3\n+test_expect_success 'git-p4 add complex branches' '\n+\ttest_when_finished cleanup_git &&\n+\ttest_create_repo \"$git\" &&\n+\t(\n+\t\tcd \"$cli\" &&\n+\t\tchangelist=$(p4 changes -m1 //depot/... | cut -d\" \" -f2) &&\n+\t\tchangelist=$(($changelist - 5)) &&\n+\t\tp4 integrate //depot/branch1/...@$changelist //depot/branch4/... &&\n+\t\tp4 submit -d \"Integrate branch4 from branch1@${changelist}\" &&\n+\t\tchangelist=$(($changelist + 2)) &&\n+\t\tp4 integrate //depot/branch1/...@$changelist //depot/branch5/... &&\n+\t\tp4 submit -d \"Integrate branch5 from branch1@${changelist}\"\n+\t)\n+'\n+\n+# Configure branches through git-config and clone them. git-p4 will only be able\n+# to clone the original structure if it is able to detect the origin changelist\n+# of each branch.\n+test_expect_success 'git-p4 clone complex branches' '\n+\ttest_when_finished cleanup_git &&\n+\ttest_create_repo \"$git\" &&\n+\t(\n+\t\tcd \"$git\" &&\n+\t\tgit config git-p4.branchList branch1:branch2 &&\n+\t\tgit config --add git-p4.branchList branch1:branch3 &&\n+\t\tgit config --add git-p4.branchList branch1:branch4 &&\n+\t\tgit config --add git-p4.branchList branch1:branch5 &&\n+\t\t\"$GITP4\" clone --dest=. --detect-branches //depot@all &&\n+\t\tgit log --all --graph --decorate --stat &&\n+\t\tgit reset --hard p4/depot/branch1 &&\n+\t\ttest_path_is_file file1 &&\n+\t\ttest_path_is_file file2 &&\n+\t\ttest_path_is_file file3 &&\n+\t\tgrep update file2 &&\n+\t\tgit reset --hard p4/depot/branch2 &&\n+\t\ttest_path_is_file file1 &&\n+\t\ttest_path_is_file file2 &&\n+\t\ttest_path_is_missing file3 &&\n+\t\ttest_must_fail grep update file2 &&\n+\t\tgit reset --hard p4/depot/branch3 &&\n+\t\ttest_path_is_file file1 &&\n+\t\ttest_path_is_file file2 &&\n+\t\ttest_path_is_file file3 &&\n+\t\tgrep update file2 &&\n+\t\tgit reset --hard p4/depot/branch4 &&\n+\t\ttest_path_is_file file1 &&\n+\t\ttest_path_is_file file2 &&\n+\t\ttest_path_is_missing file3 &&\n+\t\ttest_must_fail grep update file2 &&\n+\t\tgit reset --hard p4/depot/branch5 &&\n+\t\ttest_path_is_file file1 &&\n+\t\ttest_path_is_file file2 &&\n+\t\ttest_path_is_file file3 &&\n+\t\ttest_must_fail grep update file2 &&\n+\t\ttest_path_is_missing .git/git-p4-tmp\n \t)\n '\n \n-- \n1.7.8.3\n"},{"id":"183104","messageId":"1327535304-11332-4-git-send-email-vitor.hda@gmail.com","threadId":"29447","inReplyTo":"1327535304-11332-1-git-send-email-vitor.hda@gmail.com","subject":"[PATCH v3 3/3] git-p4: Change p4 command invocation","fromName":"Vitor Antunes","fromEmail":"vitor.hda@gmail.com","sentAt":"2012-01-25T23:48:24Z","receivedAt":"2012-01-25T23:48:24Z","isPatch":true,"sender":{"key":"vitor.hda@gmail.com","avatar":"https://avatars.githubusercontent.com/u/606876?v=4"},"body":"From: Pete Wyckoff <pw@padd.com>\n\nChange p4 command invocation to avoid going through the shell. This\nallows names with spaces and wildcards to work.\n\nSigned-off-by: Pete Wyckoff <pw@padd.com>\nSigned-off-by: Vitor Antunes <vitor.hda@gmail.com>\n---\n contrib/fast-import/git-p4 |    2 +-\n 1 files changed, 1 insertions(+), 1 deletions(-)\n\ndiff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4\nindex 584cc41..74d3613 100755\n--- a/contrib/fast-import/git-p4\n+++ b/contrib/fast-import/git-p4\n@@ -1975,7 +1975,7 @@ class P4Sync(Command, P4UserMap):\n     def importChanges(self, changes):\n         cnt = 1\n         for change in changes:\n-            description = p4Cmd(\"describe %s\" % change)\n+            description = p4Cmd([\"describe\", str(change)])\n             self.updateOptionDict(description)\n \n             if not self.silent:\n-- \n1.7.8.3\n"},{"id":"183108","messageId":"20120126042149.GA24269@padd.com","threadId":"29447","inReplyTo":"1327535304-11332-1-git-send-email-vitor.hda@gmail.com","subject":"Re: [PATCH v3 0/3] git-p4: Search for parent commit on branch creation","fromName":"Pete Wyckoff","fromEmail":"pw@padd.com","sentAt":"2012-01-26T04:21:49Z","receivedAt":"2012-01-26T04:21:49Z","isPatch":true,"sender":{"key":"pw@padd.com","avatar":null},"body":"vitor.hda@gmail.com wrote on Wed, 25 Jan 2012 23:48 +0000:\n> I think this will, hopefully, be the final version of this series of\n> patches. This version includes the following changes since v2:\n> \n>  - Move search algorithm into its own function.\n>  - Use lists instead of strings on shell commands.\n>  - Some small (almost cosmetic) updates to test cases.\n\nWhole series\n\nAcked-by: Pete Wyckoff <pw@padd.com>\n\nThanks for making all the changes.\n\n> Pete Wyckoff (1):\n>   git-p4: Change p4 command invocation\n> \n> Vitor Antunes (2):\n>   git-p4: Search for parent commit on branch creation\n>   git-p4: Add test case for complex branch import\n> \n>  contrib/fast-import/git-p4 |   48 +++++++++++++++++++++-\n>  t/t9801-git-p4-branch.sh   |   94 ++++++++++++++++++++++++++++++++++++++++---\n>  2 files changed, 133 insertions(+), 9 deletions(-)\n> \n> -- \n> 1.7.8.3\n> \n"}]}