[PATCH] git-p4: properly initialize the largefiles

From: Bertin Philippe via GitGitGadget <gitgitgadget@gmail.com>
Date: Apr 26, 2022, 18:59 UTC
Message-ID: <pull.1253.git.git.1650999569737.gitgitgadget@gmail.com>

From: tumata <tumata.codes@gmail.com>

This commit takes care of initializing the largeFiles with the already existing files referenced in the .gitattributes instead of initializing with an empty set by default.

This makes it so that the 'git p4 sync' command works as expected with git-lfs.

Signed-off-by: tumata <tumata.codes@gmail.com>
---
    git-p4: Properly initialize the largeFiles set so that it includes the
    already existing large files
    
    git-p4 sync isn't taking into account existing files that are managed by
    git-lfs. This is fine when doing git p4 clone but doing git p4 sync
    needs to take into account the files that are already managed by
    git-lfs.
    
    This change reads-in the .gitattributes and re-generate the data so that
    sync can iteratively add files to the git-lfs. It takes care of
    initializing the largeFiles with the already existing files referenced
    in the .gitattributes instead of initializing with an empty set by
    default.
    
    We had issues cloning the first commit, then syncing the remaining
    commits of a really large Perforce repo (making hundreds of smaller sync
    instead of one massive sync). The sync was deleting the git-lfs each
    time. We were able to fix this issue by first reading the current
    content of git-lfs instead of initializing to an empty set.
    
    Signed-off-by: tumata 5684571+tumata@users.noreply.github.com

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1253%2Ftumata%2Fgit-p4-initialize-large-files-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1253/tumata/git-p4-initialize-large-files-v1
Pull-Request: https://github.com/git/git/pull/1253

 git-p4.py | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/git-p4.py b/git-p4.py
index a9b1f904410..576e923a1de 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -1195,9 +1195,28 @@ class LargeFileSystem(object):
     """Base class for large file system support."""
 
     def __init__(self, writeToGitStream):
-        self.largeFiles = set()
+        self.largeFiles = self.parseLargeFiles()
         self.writeToGitStream = writeToGitStream
 
+    def parseLargeFiles(self):
+        """Parse large files in order to initially populate 'largeFiles'"""
+        paths = set()
+        try:
+            cmd = ['git', 'show', 'p4/master:.gitattributes']
+            p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
+            if p.returncode:
+                print("Failed to read .gitattributed - error code: " + p.returncode)
+                raise
+            out = p.stdout.readlines()
+            for line in out:
+                if line.startswith('/'):
+                    path = line[1:].split(' ', 1)[0]
+                    path = path.replace('[[:space:]]', ' ')
+                    paths.add(path)
+        except:
+            print("parseLargeFiles: .gitattributes does not appear to exist.")
+        return paths
+
     def generatePointer(self, cloneDestination, contentFile):
         """Return the content of a pointer file that is stored in Git instead of
            the actual content."""

base-commit: 6cd33dceed60949e2dbc32e3f0f5e67c4c882e1e

-- 
gitgitgadget