git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC PATCH 1/2] git-p4: introduce alien branch mappings

From
MAMazo, Andrey <amazo@checkvideo.com>
Date
Mar 22, 2019, 19:54 UTC
Message-ID
<b02df749b9266ac8c73707617a171122156621ab.1553283214.git.amazo@checkvideo.com>
In-Reply-To
<cover.1553283214.git.amazo@checkvideo.com>
Labels in Perforce are not global, but can be placed on a particular view/subdirectory.
This might pose difficulties when importing only parts of Perforce depot into a git repository.
For example:
 1. Depot layout is as follows:
    //depot/metaproject/branch1/subprojectA/...
    //depot/metaproject/branch1/subprojectB/...
    //depot/metaproject/branch2/subprojectA/...
    //depot/metaproject/branch2/subprojectB/...
 2. Labels are placed as follows:
    * label 1A on //depot/metaproject/branch1/subprojectA/...
    * label 1B on //depot/metaproject/branch1/subprojectB/...
    * label 2A on //depot/metaproject/branch2/subprojectA/...
    * label 2B on //depot/metaproject/branch2/subprojectB/...
 3. The goal is to import
    subprojectA into subprojectA.git and
    subprojectB into subprojectB.git
    preserving all the branches and labels.
 4. Importing subprojectA.
    Label 1A is imported fine because it's placed on certain commit on branch1.
    However, label 1B is not imported because it's placed on a commit in another subproject:
    git-p4 says: "importing label 1B: could not find git commit for changelist ..."
    The same is with label 2A, which is imported; and 2B, which is not.

Currently, there is no easy way (that I'm aware of) to tell git-p4 to import an empty commit into a desired branch, so that a label placed on that changelist could be imported as well, It might be possible to get a similar effect by importing both subprojectA and B in a single git repo, and then running `git filter-branch --subdirectory-filter subprojectA`, but this might produce way more irrelevant empty commits, than needed for labels. (although imported changelists can be limited with git-p4 --changesfile option)

So, introduce a concept of an "alien" branch.
In the above example of importing subprojectA,
 * branch1/subprojectB is an alien branch for branch1/subprojectA;
 * branch2/subprojectB is an alien branch for branch2/subprojectA.
Any changelist for branch1/subprojectB will be imported into subprojectA.git branch1
as an empty commit for the sole purpose of being labeled with a tag later
or just to preserve the history of changes across the branches.

This relation between branches is specified in a similar way to branchList: `git config --add git-p4.alienLabelBranchMap alien_branch:real_branch`

For the example of importing subprojectA above, config parameters are ``` git config --add git-p4.branchList branch1/subprojectA:branch2/subprojectA git config --add git-p4.alienLabelBranchMap branch1/subprojectB:branch1/subprojectA git config --add git-p4.alienLabelBranchMap branch2/subprojectB:branch2/subprojectA ```

A similar use case, is when a label is placed on a changelist for an excluded path.
 1. Depot layout is as follows:
    //depot/branch1/...
    //depot/branch1/exclude_me/...
 2. Labels are placed as follows:
    * label 1  on //depot/branch1/...
    * label 1E on //depot/branch1/exclude_me/...
 3. The goal is to import
    //depot/... into depot.git excluding files under
    //depot/branch1/exclude_me/...
    and preserving all the branches and labels.
 4. Importing subprojectA.
    Label 1 is imported fine because it's placed on certain commit on branch1.
    However, label 1E is not imported because it's placed on a commit which is excluded.

For this use case, the config would be ``` git config --add git-p4.alienLabelBranchMap branch1/exclude_me:branch1 ```

Note that the current implementation doesn't process alien branches when a clientspec is used.

Diff best viewed with --ignore-all-space .
Signed-off-by: Andrey Mazo <amazo@checkvideo.com>
---
Notes:
    Documentation changes and tests are obviously missing,
    but I hoped to get some feedback on the idea overall
    before working on those.
    
    A better name for "alien" branches is very welcome.
 git-p4.py | 59 ++++++++++++++++++++++++++++++++++++++++++++-----------
 1 file changed, 47 insertions(+), 12 deletions(-)
diff --git a/git-p4.py b/git-p4.py
index 148ea6f1b0..40bc84573b 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -2610,10 +2610,12 @@ def __init__(self):
 
         # map from branch depot path to parent branch
         self.knownBranches = {}
         self.initialParents = {}
 
+        self.knownAlienLabelBranches = {}
+
         self.tz = "%+03d%02d" % (- time.timezone / 3600, ((- time.timezone % 3600) / 60))
         self.labels = {}
 
     # Force a checkpoint in fast-import and wait for it to finish
     def checkpoint(self):
@@ -2705,14 +2707,37 @@ def splitFilesIntoBranches(self, commit):
         if self.clientSpecDirs:
             files = self.extractFilesFromCommit(commit)
             self.clientSpecDirs.update_client_spec_path_cache(files)
 
         branches = {}
+        alienBranches = {}
         fnum = 0
         while "depotFile%s" % fnum in commit:
             path =  commit["depotFile%s" % fnum]
             found = self.isPathWanted(path)
+
+            # start with the full relative path where this file would
+            # go in a p4 client
+            if self.useClientSpec:
+                # with a clientspec, we won't know where the file goes if it's excluded
+                if found:
+                    relPath = self.clientSpecDirs.map_in_client(path)
+                else:
+                    relPath = None
+            else:
+                # without a clientspec, we can guess a file path even if it's excluded
+                relPath = self.stripRepoPath(path, self.depotPaths)
+
+            # Process alien branches before excludes (i.e. before respecting `found`) --
+            # picking (empty) commits from excluded paths is one of the use-cases for alien branches.
+            # We don't commit any files for alien branches, so don't violate excludes.
+            for (alienPath, ourPath) in self.knownAlienLabelBranches.items():
+                if relPath is not None and p4PathStartsWith(relPath, alienPath + '/') and ourPath in self.knownBranches:
+                    # we don't put any files since they are under the paths, we're not interested in.
+                    # however, we still want the commit message and etc.
+                    alienBranches[ourPath] = [{"alienPath": alienPath}]
+
             if not found:
                 fnum = fnum + 1
                 continue
 
             file = {}
@@ -2720,26 +2745,23 @@ def splitFilesIntoBranches(self, commit):
             file["rev"] = commit["rev%s" % fnum]
             file["action"] = commit["action%s" % fnum]
             file["type"] = commit["type%s" % fnum]
             fnum = fnum + 1
 
-            # start with the full relative path where this file would
-            # go in a p4 client
-            if self.useClientSpec:
-                relPath = self.clientSpecDirs.map_in_client(path)
-            else:
-                relPath = self.stripRepoPath(path, self.depotPaths)
-
             for branch in self.knownBranches.keys():
                 # add a trailing slash so that a commit into qt/4.2foo
                 # doesn't end up in qt/4.2, e.g.
                 if p4PathStartsWith(relPath, branch + "/"):
                     if branch not in branches:
                         branches[branch] = []
                     branches[branch].append(file)
                     break
 
+        # not on any branch, check out our alien mapping
+        if not branches:
+            branches = alienBranches
+
         return branches
 
     def writeToGitStream(self, gitMode, relPath, contents):
         self.gitStream.write('M %s inline %s\n' % (gitMode, relPath))
         self.gitStream.write('data %d\n' % sum(len(d) for d in contents))
@@ -3283,10 +3305,17 @@ def getBranchMappingFromGitBranches(self):
                 branch = "main"
             else:
                 branch = branch[len(self.projectName):]
             self.knownBranches[branch] = branch
 
+    def getAlienLabelBranchMapping(self):
+        alienLabelBranches = gitConfigList("git-p4.alienLabelBranchMap")
+        for mapping in alienLabelBranches:
+            if mapping:
+                (alien, ours) = mapping.split(":")
+                self.knownAlienLabelBranches[alien] = ours
+
     def updateOptionDict(self, d):
         option_keys = {}
         if self.keepRepoPath:
             option_keys['keepRepoPath'] = 1
 
@@ -3396,17 +3425,22 @@ def importChanges(self, changes, origin_revision=0):
 
             try:
                 if self.detectBranches:
                     branches = self.splitFilesIntoBranches(description)
                     for branch in branches.keys():
-                        ## HACK  --hwn
-                        branchPrefix = self.depotPaths[0] + branch + "/"
-                        self.branchPrefixes = [ branchPrefix ]
+                        # hack branch prefix and file list for alien branches
+                        if branches[branch] and "alienPath" in branches[branch][0]:
+                            branchPrefix = self.depotPaths[0] + branches[branch][0]["alienPath"] + "/"
+                            filesForCommit = []
+                        else:
+                            ## HACK  --hwn
+                            branchPrefix = self.depotPaths[0] + branch + "/"
+                            filesForCommit = branches[branch]
 
-                        parent = ""
+                        self.branchPrefixes = [branchPrefix]
 
-                        filesForCommit = branches[branch]
+                        parent = ""
 
                         if self.verbose:
                             print("branch is %s" % branch)
 
                         self.updatedBranches.add(branch)
@@ -3715,10 +3749,11 @@ def run(self, args):
 
             if self.hasOrigin:
                 self.getBranchMappingFromGitBranches()
             else:
                 self.getBranchMapping()
+                self.getAlienLabelBranchMapping()
             if self.verbose:
                 print("p4-git branches: %s" % self.p4BranchesInGit)
                 print("initial parents: %s" % self.initialParents)
             for b in self.p4BranchesInGit:
                 if b != "master":

-- 
2.19.2
Previous: Mazo, AndreyNext: Luke Diamand
Message 18 of 40 in “git-p4: a few assorted fixes for branches, excludes”
  1. 0/5 git-p4: a few assorted fixes for branches, excludesMazo, Andrey, Mar 4, 2019
  2. 1/5 git-p4: detect/prevent infinite loop in gitCommitByP4Change()Mazo, Andrey, Mar 4, 2019
  3. 2/5 git-p4: match branches case insensitively if configuredMazo, Andrey, Mar 4, 2019
  4. 3/5 git-p4: don't groom exclude path list on every commitMazo, Andrey, Mar 4, 2019
  5. 4/5 git-p4: add failing test for "don't exclude other files with same prefix"Mazo, Andrey, Mar 4, 2019
  6. 5/5 git-p4: don't exclude other files with same prefixMazo, Andrey, Mar 4, 2019
  7. 0/7 git-p4: a few assorted fixes for branches, excludesMazo, Andrey, Mar 21, 2019
  8. 2/7 git-p4: match branches case insensitively if configuredMazo, Andrey, Mar 21, 2019
  9. Luke DiamandMar 23, 2019
  10. Mazo, AndreyMar 25, 2019
  11. 1/7 git-p4: detect/prevent infinite loop in gitCommitByP4Change()Mazo, Andrey, Mar 21, 2019
  12. 3/7 git-p4: don't groom exclude path list on every commitMazo, Andrey, Mar 21, 2019
  13. 4/7 git-p4: add failing test for "don't exclude other files with same prefix"Mazo, Andrey, Mar 21, 2019
  14. 5/7 git-p4: don't exclude other files with same prefixMazo, Andrey, Mar 21, 2019
  15. 6/7 git-p4: add failing test for "git-p4: respect excluded paths when detecting branches"Mazo, Andrey, Mar 21, 2019
  16. 7/7 git-p4: respect excluded paths when detecting branchesMazo, Andrey, Mar 21, 2019
  17. 0/2 git-p4: "alien" branches and load changelist info from fileMazo, Andrey, Mar 22, 2019
  18. 1/2 git-p4: introduce alien branch mappingsMazo, Andrey, Mar 22, 2019
  19. Luke DiamandMar 23, 2019
  20. Mazo, AndreyMar 26, 2019
  21. 1/1 git-p4: inexact label detectionMazo, Andrey, Mar 27, 2019
  22. 2/2 git-p4: support loading changelist descriptions from filesMazo, Andrey, Mar 22, 2019
  23. Luke DiamandMar 23, 2019
  24. Mazo, AndreyMar 25, 2019
  25. 0/8 git-p4: a few assorted fixes for branches, excludesMazo, Andrey, Apr 1, 2019
  26. 1/8 git-p4: detect/prevent infinite loop in gitCommitByP4Change()Mazo, Andrey, Apr 1, 2019
  27. 2/8 git-p4: add failing test for "git-p4: match branches case insensitively if configured"Mazo, Andrey, Apr 1, 2019
  28. SZEDER GáborApr 2, 2019
  29. Mazo, AndreyApr 2, 2019
  30. Junio C HamanoApr 3, 2019
  31. 4/8 git-p4: don't groom exclude path list on every commitMazo, Andrey, Apr 1, 2019
  32. 3/8 git-p4: match branches case insensitively if configuredMazo, Andrey, Apr 1, 2019
  33. 5/8 git-p4: add failing test for "don't exclude other files with same prefix"Mazo, Andrey, Apr 1, 2019
  34. 6/8 git-p4: don't exclude other files with same prefixMazo, Andrey, Apr 1, 2019
  35. 7/8 git-p4: add failing test for "git-p4: respect excluded paths when detecting branches"Mazo, Andrey, Apr 1, 2019
  36. 8/8 git-p4: respect excluded paths when detecting branchesMazo, Andrey, Apr 1, 2019
  37. Mazo, AndreyApr 1, 2019
  38. 0/2 git-p4: inexact labels and load changelist description from fileMazo, Andrey, Apr 2, 2019
  39. 1/2 git-p4: inexact label detectionMazo, Andrey, Apr 2, 2019
  40. 2/2 git-p4: support loading changelist descriptions from filesMazo, Andrey, Apr 2, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.