{"thread":{"id":"40399","subject":"[PATCH v7 5/7] git-p4: check free space during streaming","startedAt":"2015-09-21T22:41:06Z","lastAt":"2015-09-25T09:09:46Z","messageCount":21,"participants":["larsxschneider@gmail.com","Junio C Hamano","Lars Schneider","Luke Diamand"],"isPatch":true,"patchVersion":7,"patchTotal":7},"messages":[{"id":"270445","messageId":"1442875273-48610-1-git-send-email-larsxschneider@gmail.com","threadId":"40399","inReplyTo":null,"subject":"[PATCH v7 0/7] git-p4: add support for large file systems","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-09-21T22:41:06Z","receivedAt":"2015-09-21T22:41:06Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider \n\ndiff to v6:\n* fix spaces in .gitattributes for Git-LFS files (old \"[:space:]\", new \"[[:space:]]\")\n* generated patch on top of next (95c4325) to resolve merge conflicts\n\nI am sorry about the \"[:space:]\" bug this late. I had the fix in my development\nbranch but missed to apply it in this patch series.\n\n@Junio:\nYour conflict resolution in pu looks good. Nevertheless, I made this patch on\ntop of next to ease the integration. I hope this is ok.\n\nCheers,\nLars\n\nLars Schneider (7):\n git-p4: add optional type specifier to gitConfig reader\n git-p4: add gitConfigInt reader\n git-p4: return an empty list if a list config has no values\n git-p4: add file streaming progress in verbose mode\n git-p4: check free space during streaming\n git-p4: add support for large file systems\n git-p4: add Git LFS backend for large file system\n\n Documentation/git-p4.txt | 32 +++++\n git-p4.py | 278 ++++++++++++++++++++++++++++++++++++++++---\n t/t9823-git-p4-mock-lfs.sh | 170 ++++++++++++++++++++++++++\n t/t9824-git-p4-git-lfs.sh | 288 +++++++++++++++++++++++++++++++++++++++++++++\n 4 files changed, 751 insertions(+), 17 deletions(-)\n create mode 100755 t/t9823-git-p4-mock-lfs.sh\n create mode 100755 t/t9824-git-p4-git-lfs.sh\n\n--\n2.5.1\n"},{"id":"270444","messageId":"1442875273-48610-2-git-send-email-larsxschneider@gmail.com","threadId":"40399","inReplyTo":"1442875273-48610-1-git-send-email-larsxschneider@gmail.com","subject":"[PATCH v7 1/7] git-p4: add optional type specifier to gitConfig reader","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-09-21T22:41:07Z","receivedAt":"2015-09-21T22:41:07Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider \n\nThe functions “gitConfig” and “gitConfigBool” are almost identical. Make “gitConfig” more generic by adding an optional type specifier. Use the type specifier “—bool” with “gitConfig” to implement “gitConfigBool. This prepares the implementation of other type specifiers such as “—int”.\n\nSigned-off-by: Lars Schneider \nAcked-by: Luke Diamand \n---\n git-p4.py | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/git-p4.py b/git-p4.py\nindex 215f556..2659079 100755\n--- a/git-p4.py\n+++ b/git-p4.py\n@@ -604,9 +604,12 @@ def gitBranchExists(branch):\n \n _gitConfig = {}\n \n-def gitConfig(key):\n+def gitConfig(key, typeSpecifier=None):\n if not _gitConfig.has_key(key):\n- cmd = [ \"git\", \"config\", key ]\n+ cmd = [ \"git\", \"config\" ]\n+ if typeSpecifier:\n+ cmd += [ typeSpecifier ]\n+ cmd += [ key ]\n s = read_pipe(cmd, ignore_error=True)\n _gitConfig[key] = s.strip()\n return _gitConfig[key]\n@@ -617,10 +620,7 @@ def gitConfigBool(key):\n in the config.\"\"\"\n \n if not _gitConfig.has_key(key):\n- cmd = [ \"git\", \"config\", \"--bool\", key ]\n- s = read_pipe(cmd, ignore_error=True)\n- v = s.strip()\n- _gitConfig[key] = v == \"true\"\n+ _gitConfig[key] = gitConfig(key, '--bool') == \"true\"\n return _gitConfig[key]\n \n def gitConfigList(key):\n-- \n2.5.1\n"},{"id":"270446","messageId":"1442875273-48610-3-git-send-email-larsxschneider@gmail.com","threadId":"40399","inReplyTo":"1442875273-48610-1-git-send-email-larsxschneider@gmail.com","subject":"[PATCH v7 2/7] git-p4: add gitConfigInt reader","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-09-21T22:41:08Z","receivedAt":"2015-09-21T22:41:08Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider \n\nAdd a git config reader for integer variables. Please note that the git config implementation automatically supports k, m, and g suffixes.\n\nSigned-off-by: Lars Schneider \n---\n git-p4.py | 11 +++++++++++\n 1 file changed, 11 insertions(+)\n\ndiff --git a/git-p4.py b/git-p4.py\nindex 2659079..2715895 100755\n--- a/git-p4.py\n+++ b/git-p4.py\n@@ -623,6 +623,17 @@ def gitConfigBool(key):\n _gitConfig[key] = gitConfig(key, '--bool') == \"true\"\n return _gitConfig[key]\n \n+def gitConfigInt(key):\n+ if not _gitConfig.has_key(key):\n+ cmd = [ \"git\", \"config\", \"--int\", key ]\n+ s = read_pipe(cmd, ignore_error=True)\n+ v = s.strip()\n+ try:\n+ _gitConfig[key] = int(gitConfig(key, '--int'))\n+ except ValueError:\n+ _gitConfig[key] = None\n+ return _gitConfig[key]\n+\n def gitConfigList(key):\n if not _gitConfig.has_key(key):\n s = read_pipe([\"git\", \"config\", \"--get-all\", key], ignore_error=True)\n-- \n2.5.1\n"},{"id":"270443","messageId":"1442875273-48610-4-git-send-email-larsxschneider@gmail.com","threadId":"40399","inReplyTo":"1442875273-48610-1-git-send-email-larsxschneider@gmail.com","subject":"[PATCH v7 3/7] git-p4: return an empty list if a list config has no values","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-09-21T22:41:09Z","receivedAt":"2015-09-21T22:41:09Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider \n\nSigned-off-by: Lars Schneider \n---\n git-p4.py | 2 ++\n 1 file changed, 2 insertions(+)\n\ndiff --git a/git-p4.py b/git-p4.py\nindex 2715895..e7d1b3f 100755\n--- a/git-p4.py\n+++ b/git-p4.py\n@@ -638,6 +638,8 @@ def gitConfigList(key):\n if not _gitConfig.has_key(key):\n s = read_pipe([\"git\", \"config\", \"--get-all\", key], ignore_error=True)\n _gitConfig[key] = s.strip().split(os.linesep)\n+ if _gitConfig[key] == ['']:\n+ _gitConfig[key] = []\n return _gitConfig[key]\n \n def p4BranchesInGit(branchesAreInRemotes=True):\n-- \n2.5.1\n"},{"id":"270449","messageId":"1442875273-48610-5-git-send-email-larsxschneider@gmail.com","threadId":"40399","inReplyTo":"1442875273-48610-1-git-send-email-larsxschneider@gmail.com","subject":"[PATCH v7 4/7] git-p4: add file streaming progress in verbose mode","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-09-21T22:41:10Z","receivedAt":"2015-09-21T22:41:10Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider \n\nIf a file is streamed from P4 to Git then the verbose mode prints\ncontinuously the progress as percentage like this:\n//depot/file.bin 20% (10 MB)\n\nUpon completion the progress is overwritten with depot source, local\nfile and size like this:\n//depot/file.bin --> local/file.bin (10 MB)\n\nSigned-off-by: Lars Schneider \n---\n git-p4.py | 20 ++++++++++++++++++--\n 1 file changed, 18 insertions(+), 2 deletions(-)\n\ndiff --git a/git-p4.py b/git-p4.py\nindex e7d1b3f..fabc900 100755\n--- a/git-p4.py\n+++ b/git-p4.py\n@@ -2171,7 +2171,9 @@ class P4Sync(Command, P4UserMap):\n def streamOneP4File(self, file, contents):\n relPath = self.stripRepoPath(file['depotFile'], self.branchPrefixes)\n if verbose:\n- sys.stderr.write(\"%s\\n\" % relPath)\n+ size = int(self.stream_file['fileSize'])\n+ sys.stdout.write('\\r%s --> %s (%i MB)\\n' % (file['depotFile'], relPath, size/1024/1024))\n+ sys.stdout.flush()\n \n (type_base, type_mods) = split_p4_type(file[\"type\"])\n \n@@ -2258,7 +2260,8 @@ class P4Sync(Command, P4UserMap):\n def streamOneP4Deletion(self, file):\n relPath = self.stripRepoPath(file['path'], self.branchPrefixes)\n if verbose:\n- sys.stderr.write(\"delete %s\\n\" % relPath)\n+ sys.stdout.write(\"delete %s\\n\" % relPath)\n+ sys.stdout.flush()\n self.gitStream.write(\"D %s\\n\" % relPath)\n \n # handle another chunk of streaming data\n@@ -2298,10 +2301,23 @@ class P4Sync(Command, P4UserMap):\n # 'data' field we need to append to our array\n for k in marshalled.keys():\n if k == 'data':\n+ if 'streamContentSize' not in self.stream_file:\n+ self.stream_file['streamContentSize'] = 0\n+ self.stream_file['streamContentSize'] += len(marshalled['data'])\n self.stream_contents.append(marshalled['data'])\n else:\n self.stream_file[k] = marshalled[k]\n \n+ if (verbose and\n+ 'streamContentSize' in self.stream_file and\n+ 'fileSize' in self.stream_file and\n+ 'depotFile' in self.stream_file):\n+ size = int(self.stream_file[\"fileSize\"])\n+ if size > 0:\n+ progress = 100*self.stream_file['streamContentSize']/size\n+ sys.stdout.write('\\r%s %d%% (%i MB)' % (self.stream_file['depotFile'], progress, int(size/1024/1024)))\n+ sys.stdout.flush()\n+\n self.stream_have_file_info = True\n \n # Stream directly from \"p4 files\" into \"git fast-import\"\n-- \n2.5.1\n"},{"id":"270442","messageId":"1442875273-48610-6-git-send-email-larsxschneider@gmail.com","threadId":"40399","inReplyTo":"1442875273-48610-1-git-send-email-larsxschneider@gmail.com","subject":"[PATCH v7 5/7] git-p4: check free space during streaming","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-09-21T22:41:11Z","receivedAt":"2015-09-21T22:41:11Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider \n\ngit-p4 will just halt if there is not enough disk space while\nstreaming content from P4 to Git. Add a check to ensure at least\n4 times (arbitrarily chosen) the size of a streamed file is available.\n\nSigned-off-by: Lars Schneider \n---\n git-p4.py | 27 ++++++++++++++++++++++++++-\n 1 file changed, 26 insertions(+), 1 deletion(-)\n\ndiff --git a/git-p4.py b/git-p4.py\nindex fabc900..3931989 100755\n--- a/git-p4.py\n+++ b/git-p4.py\n@@ -104,6 +104,16 @@ def chdir(path, is_client_path=False):\n path = os.getcwd()\n os.environ['PWD'] = path\n \n+def calcDiskFree(dirname):\n+ \"\"\"Return free space in bytes on the disk of the given dirname.\"\"\"\n+ if platform.system() == 'Windows':\n+ free_bytes = ctypes.c_ulonglong(0)\n+ ctypes.windll.kernel32.GetDiskFreeSpaceExW(ctypes.c_wchar_p(dirname), None, None, ctypes.pointer(free_bytes))\n+ return free_bytes.value\n+ else:\n+ st = os.statvfs(dirname)\n+ return st.f_bavail * st.f_frsize\n+\n def die(msg):\n if verbose:\n raise Exception(msg)\n@@ -2045,6 +2055,14 @@ class P4Sync(Command, P4UserMap):\n self.clientSpecDirs = None\n self.tempBranches = []\n self.tempBranchLocation = \"git-p4-tmp\"\n+ self.largeFileSystem = None\n+ self.cloneDestination = None\n+\n+ if gitConfig('git-p4.largeFileSystem'):\n+ largeFileSystemConstructor = globals()[gitConfig('git-p4.largeFileSystem')]\n+ self.largeFileSystem = largeFileSystemConstructor(\n+ lambda git_mode, relPath, contents: self.writeToGitStream(git_mode, relPath, contents)\n+ )\n \n if gitConfig(\"git-p4.syncFromOrigin\") == \"false\":\n self.syncWithOrigin = False\n@@ -2273,6 +2291,14 @@ class P4Sync(Command, P4UserMap):\n if marshalled[\"code\"] == \"error\":\n if \"data\" in marshalled:\n err = marshalled[\"data\"].rstrip()\n+\n+ if not err and self.cloneDestination and 'fileSize' in self.stream_file:\n+ required_bytes = int((4 * int(self.stream_file[\"fileSize\"])) - calcDiskFree(self.cloneDestination))\n+ if required_bytes > 0:\n+ err = 'Not enough space left on %s! Free at least %i MB.' % (\n+ os.path.abspath(self.cloneDestination), required_bytes/1024/1024\n+ )\n+\n if err:\n f = None\n if self.stream_have_file_info:\n@@ -3244,7 +3270,6 @@ class P4Clone(P4Sync):\n optparse.make_option(\"--bare\", dest=\"cloneBare\",\n action=\"store_true\", default=False),\n ]\n- self.cloneDestination = None\n self.needsGit = False\n self.cloneBare = False\n \n-- \n2.5.1\n"},{"id":"270448","messageId":"1442875273-48610-7-git-send-email-larsxschneider@gmail.com","threadId":"40399","inReplyTo":"1442875273-48610-1-git-send-email-larsxschneider@gmail.com","subject":"[PATCH v7 6/7] git-p4: add support for large file systems","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-09-21T22:41:12Z","receivedAt":"2015-09-21T22:41:12Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider \n\nPerforce repositories can contain large (binary) files. Migrating these\nrepositories to Git generates very large local clones. External storage\nsystems such as Git LFS [1], Git Fat [2], Git Media [3], git-annex [4]\ntry to address this problem.\n\nAdd a generic mechanism to detect large files based on extension,\nuncompressed size, and/or compressed size.\n\n[1] https://git-lfs.github.com/\n[2] https://github.com/jedbrown/git-fat\n[3] https://github.com/alebedev/git-media\n[4] https://git-annex.branchable.com/\n\nSigned-off-by: Lars Schneider \n---\n Documentation/git-p4.txt | 32 +++++++++\n git-p4.py | 138 +++++++++++++++++++++++++++++++++---\n t/t9823-git-p4-mock-lfs.sh | 170 +++++++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 330 insertions(+), 10 deletions(-)\n create mode 100755 t/t9823-git-p4-mock-lfs.sh\n\ndiff --git a/Documentation/git-p4.txt b/Documentation/git-p4.txt\nindex 12a57d4..19833e6 100644\n--- a/Documentation/git-p4.txt\n+++ b/Documentation/git-p4.txt\n@@ -517,6 +517,38 @@ git-p4.pathEncoding::\n \tto transcode the paths to UTF-8. As an example, Perforce on Windows\n \toften uses “cp1252” to encode path names.\n \n+git-p4.largeFileSystem::\n+\tSpecify the system that is used for large (binary) files. Please note\n+\tthat large file systems do not support the 'git p4 submit' command.\n+\tOnly Git LFS [1] is implemented right now. Download\n+\tand install the Git LFS command line extension to use this option\n+\tand configure it like this:\n++\n+-------------\n+git config git-p4.largeFileSystem GitLFS\n+-------------\n++\n+\t[1] https://git-lfs.github.com/\n+\n+git-p4.largeFileExtensions::\n+\tAll files matching a file extension in the list will be processed\n+\tby the large file system. Do not prefix the extensions with '.'.\n+\n+git-p4.largeFileThreshold::\n+\tAll files with an uncompressed size exceeding the threshold will be\n+\tprocessed by the large file system. By default the threshold is\n+\tdefined in bytes. Add the suffix k, m, or g to change the unit.\n+\n+git-p4.largeFileCompressedThreshold::\n+\tAll files with a compressed size exceeding the threshold will be\n+\tprocessed by the large file system. This option might slow down\n+\tyour clone/sync process. By default the threshold is defined in\n+\tbytes. Add the suffix k, m, or g to change the unit.\n+\n+git-p4.pushLargeFiles::\n+\tBoolean variable which defines if large files are automatically\n+\tpushed to a server.\n+\n Submit variables\n ~~~~~~~~~~~~~~~~\n git-p4.detectRenames::\ndiff --git a/git-p4.py b/git-p4.py\nindex 3931989..7a80ebd 100755\n--- a/git-p4.py\n+++ b/git-p4.py\n@@ -22,6 +22,8 @@ import platform\n import re\n import shutil\n import stat\n+import zipfile\n+import zlib\n \n try:\n from subprocess import CalledProcessError\n@@ -932,6 +934,110 @@ def wildcard_present(path):\n m = re.search(\"[*#@%]\", path)\n return m is not None\n \n+class LargeFileSystem(object):\n+ \"\"\"Base class for large file system support.\"\"\"\n+\n+ def __init__(self, writeToGitStream):\n+ self.largeFiles = set()\n+ self.writeToGitStream = writeToGitStream\n+\n+ def generatePointer(self, cloneDestination, contentFile):\n+ \"\"\"Return the content of a pointer file that is stored in Git instead of\n+ the actual content.\"\"\"\n+ assert False, \"Method 'generatePointer' required in \" + self.__class__.__name__\n+\n+ def pushFile(self, localLargeFile):\n+ \"\"\"Push the actual content which is not stored in the Git repository to\n+ a server.\"\"\"\n+ assert False, \"Method 'pushFile' required in \" + self.__class__.__name__\n+\n+ def hasLargeFileExtension(self, relPath):\n+ return reduce(\n+ lambda a, b: a or b,\n+ [relPath.endswith('.' + e) for e in gitConfigList('git-p4.largeFileExtensions')],\n+ False\n+ )\n+\n+ def generateTempFile(self, contents):\n+ contentFile = tempfile.NamedTemporaryFile(prefix='git-p4-large-file', delete=False)\n+ for d in contents:\n+ contentFile.write(d)\n+ contentFile.close()\n+ return contentFile.name\n+\n+ def exceedsLargeFileThreshold(self, relPath, contents):\n+ if gitConfigInt('git-p4.largeFileThreshold'):\n+ contentsSize = sum(len(d) for d in contents)\n+ if contentsSize > gitConfigInt('git-p4.largeFileThreshold'):\n+ return True\n+ if gitConfigInt('git-p4.largeFileCompressedThreshold'):\n+ contentsSize = sum(len(d) for d in contents)\n+ if contentsSize <= gitConfigInt('git-p4.largeFileCompressedThreshold'):\n+ return False\n+ contentTempFile = self.generateTempFile(contents)\n+ compressedContentFile = tempfile.NamedTemporaryFile(prefix='git-p4-large-file', delete=False)\n+ zf = zipfile.ZipFile(compressedContentFile.name, mode='w')\n+ zf.write(contentTempFile, compress_type=zipfile.ZIP_DEFLATED)\n+ zf.close()\n+ compressedContentsSize = zf.infolist()[0].compress_size\n+ os.remove(contentTempFile)\n+ os.remove(compressedContentFile.name)\n+ if compressedContentsSize > gitConfigInt('git-p4.largeFileCompressedThreshold'):\n+ return True\n+ return False\n+\n+ def addLargeFile(self, relPath):\n+ self.largeFiles.add(relPath)\n+\n+ def removeLargeFile(self, relPath):\n+ self.largeFiles.remove(relPath)\n+\n+ def isLargeFile(self, relPath):\n+ return relPath in self.largeFiles\n+\n+ def processContent(self, cloneDestination, git_mode, relPath, contents):\n+ \"\"\"Processes the content of git fast import. This method decides if a\n+ file is stored in the large file system and handles all necessary\n+ steps.\"\"\"\n+ if self.exceedsLargeFileThreshold(relPath, contents) or self.hasLargeFileExtension(relPath):\n+ contentTempFile = self.generateTempFile(contents)\n+ (git_mode, contents, localLargeFile) = self.generatePointer(cloneDestination, contentTempFile)\n+\n+ # Move temp file to final location in large file system\n+ largeFileDir = os.path.dirname(localLargeFile)\n+ if not os.path.isdir(largeFileDir):\n+ os.makedirs(largeFileDir)\n+ shutil.move(contentTempFile, localLargeFile)\n+ self.addLargeFile(relPath)\n+ if gitConfigBool('git-p4.pushLargeFiles'):\n+ self.pushFile(localLargeFile)\n+ if verbose:\n+ sys.stderr.write(\"%s moved to large file system (%s)\\n\" % (relPath, localLargeFile))\n+ return (git_mode, contents)\n+\n+class MockLFS(LargeFileSystem):\n+ \"\"\"Mock large file system for testing.\"\"\"\n+\n+ def generatePointer(self, cloneDestination, contentFile):\n+ \"\"\"The pointer content is the original content prefixed with \"pointer-\".\n+ The local filename of the large file storage is derived from the file content.\n+ \"\"\"\n+ with open(contentFile, 'r') as f:\n+ content = next(f)\n+ gitMode = '100644'\n+ pointerContents = 'pointer-' + content\n+ localLargeFile = os.path.join(cloneDestination, '.git', 'mock-storage', 'local', content[:-1])\n+ return (gitMode, pointerContents, localLargeFile)\n+\n+ def pushFile(self, localLargeFile):\n+ \"\"\"The remote filename of the large file storage is the same as the local\n+ one but in a different directory.\n+ \"\"\"\n+ remotePath = os.path.join(os.path.dirname(localLargeFile), '..', 'remote')\n+ if not os.path.exists(remotePath):\n+ os.makedirs(remotePath)\n+ shutil.copyfile(localLargeFile, os.path.join(remotePath, os.path.basename(localLargeFile)))\n+\n class Command:\n def __init__(self):\n self.usage = \"usage: %prog [options]\"\n@@ -1105,6 +1211,9 @@ class P4Submit(Command, P4UserMap):\n self.p4HasMoveCommand = p4_has_move_command()\n self.branch = None\n \n+ if gitConfig('git-p4.largeFileSystem'):\n+ die(\"Large file system not supported for git-p4 submit command. Please remove it from config.\")\n+\n def check(self):\n if len(p4CmdList(\"opened ...\")) > 0:\n die(\"You have files opened with perforce! Close them before starting the sync.\")\n@@ -2064,6 +2173,12 @@ class P4Sync(Command, P4UserMap):\n lambda git_mode, relPath, contents: self.writeToGitStream(git_mode, relPath, contents)\n )\n \n+ if gitConfig('git-p4.largeFileSystem'):\n+ largeFileSystemConstructor = globals()[gitConfig('git-p4.largeFileSystem')]\n+ self.largeFileSystem = largeFileSystemConstructor(\n+ lambda git_mode, relPath, contents: self.writeToGitStream(git_mode, relPath, contents)\n+ )\n+\n if gitConfig(\"git-p4.syncFromOrigin\") == \"false\":\n self.syncWithOrigin = False\n \n@@ -2183,6 +2298,13 @@ class P4Sync(Command, P4UserMap):\n \n return branches\n \n+ def writeToGitStream(self, gitMode, relPath, contents):\n+ self.gitStream.write('M %s inline %s\\n' % (gitMode, relPath))\n+ self.gitStream.write('data %d\\n' % sum(len(d) for d in contents))\n+ for d in contents:\n+ self.gitStream.write(d)\n+ self.gitStream.write('\\n')\n+\n # output one file from the P4 stream\n # - helper for streamP4Files\n \n@@ -2263,17 +2385,10 @@ class P4Sync(Command, P4UserMap):\n if self.verbose:\n print 'Path with non-ASCII characters detected. Used %s to encode: %s ' % (encoding, relPath)\n \n- self.gitStream.write(\"M %s inline %s\\n\" % (git_mode, relPath))\n+ if self.largeFileSystem:\n+ (git_mode, contents) = self.largeFileSystem.processContent(self.cloneDestination, git_mode, relPath, contents)\n \n- # total length...\n- length = 0\n- for d in contents:\n- length = length + len(d)\n-\n- self.gitStream.write(\"data %d\\n\" % length)\n- for d in contents:\n- self.gitStream.write(d)\n- self.gitStream.write(\"\\n\")\n+ self.writeToGitStream(git_mode, relPath, contents)\n \n def streamOneP4Deletion(self, file):\n relPath = self.stripRepoPath(file['path'], self.branchPrefixes)\n@@ -2282,6 +2397,9 @@ class P4Sync(Command, P4UserMap):\n sys.stdout.flush()\n self.gitStream.write(\"D %s\\n\" % relPath)\n \n+ if self.largeFileSystem and self.largeFileSystem.isLargeFile(relPath):\n+ self.largeFileSystem.removeLargeFile(relPath)\n+\n # handle another chunk of streaming data\n def streamP4FilesCb(self, marshalled):\n \ndiff --git a/t/t9823-git-p4-mock-lfs.sh b/t/t9823-git-p4-mock-lfs.sh\nnew file mode 100755\nindex 0000000..5684ee3\n--- /dev/null\n+++ b/t/t9823-git-p4-mock-lfs.sh\n@@ -0,0 +1,170 @@\n+#!/bin/sh\n+\n+test_description='Clone repositories and store files in Mock LFS'\n+\n+. ./lib-git-p4.sh\n+\n+test_file_is_not_in_mock_lfs () {\n+\tFILE=\"$1\" &&\n+\tCONTENT=\"$2\" &&\n+\techo \"$CONTENT\" >expect_content &&\n+\ttest_path_is_file \"$FILE\" &&\n+\ttest_cmp expect_content \"$FILE\"\n+}\n+\n+test_file_is_in_mock_lfs () {\n+\tFILE=\"$1\" &&\n+\tCONTENT=\"$2\" &&\n+\tLOCAL_STORAGE=\".git/mock-storage/local/$CONTENT\" &&\n+\tSERVER_STORAGE=\".git/mock-storage/remote/$CONTENT\" &&\n+\techo \"pointer-$CONTENT\" >expect_pointer &&\n+\techo \"$CONTENT\" >expect_content &&\n+\ttest_path_is_file \"$FILE\" &&\n+\ttest_path_is_file \"$LOCAL_STORAGE\" &&\n+\ttest_path_is_file \"$SERVER_STORAGE\" &&\n+\ttest_cmp expect_pointer \"$FILE\" &&\n+\ttest_cmp expect_content \"$LOCAL_STORAGE\" &&\n+\ttest_cmp expect_content \"$SERVER_STORAGE\"\n+}\n+\n+test_file_is_deleted_in_mock_lfs () {\n+\tFILE=\"$1\" &&\n+\tCONTENT=\"$2\" &&\n+\tLOCAL_STORAGE=\".git/mock-storage/local/$CONTENT\" &&\n+\tSERVER_STORAGE=\".git/mock-storage/remote/$CONTENT\" &&\n+\techo \"pointer-$CONTENT\" >expect_pointer &&\n+\techo \"$CONTENT\" >expect_content &&\n+\t! test_path_is_file \"$FILE\" &&\n+\ttest_path_is_file \"$LOCAL_STORAGE\" &&\n+\ttest_path_is_file \"$SERVER_STORAGE\" &&\n+\ttest_cmp expect_content \"$LOCAL_STORAGE\" &&\n+\ttest_cmp expect_content \"$SERVER_STORAGE\"\n+}\n+\n+test_file_count_in_dir () {\n+\tDIR=\"$1\" &&\n+\tEXPECTED_COUNT=\"$2\" &&\n+\tfind \"$DIR\" -type f >actual &&\n+\ttest_line_count = $EXPECTED_COUNT actual\n+}\n+\n+test_expect_success 'start p4d' '\n+\tstart_p4d\n+'\n+\n+test_expect_success 'Create repo with binary files' '\n+\tclient_view \"//depot/... //client/...\" &&\n+\t(\n+\t\tcd \"$cli\" &&\n+\n+\t\techo \"content 1 txt 23 bytes\" >file1.txt &&\n+\t\tp4 add file1.txt &&\n+\t\techo \"content 2-3 bin 25 bytes\" >file2.dat &&\n+\t\tp4 add file2.dat &&\n+\t\tp4 submit -d \"Add text and binary file\" &&\n+\n+\t\tmkdir \"path with spaces\" &&\n+\t\techo \"content 2-3 bin 25 bytes\" >\"path with spaces/file3.bin\" &&\n+\t\tp4 add \"path with spaces/file3.bin\" &&\n+\t\tp4 submit -d \"Add another binary file with same content and spaces in path\" &&\n+\n+\t\techo \"content 4 bin 26 bytes XX\" >file4.bin &&\n+\t\tp4 add file4.bin &&\n+\t\tp4 submit -d \"Add another binary file with different content\"\n+\t)\n+'\n+\n+test_expect_success 'Store files in Mock LFS based on size (>24 bytes)' '\n+\tclient_view \"//depot/... //client/...\" &&\n+\ttest_when_finished cleanup_git &&\n+\t(\n+\t\tcd \"$git\" &&\n+\t\tgit init . &&\n+\t\tgit config git-p4.useClientSpec true &&\n+\t\tgit config git-p4.largeFileSystem MockLFS &&\n+\t\tgit config git-p4.largeFileThreshold 24 &&\n+\t\tgit config git-p4.pushLargeFiles True &&\n+\t\tgit p4 clone --destination=\"$git\" //depot@all &&\n+\n+\t\ttest_file_is_not_in_mock_lfs file1.txt \"content 1 txt 23 bytes\" &&\n+\t\ttest_file_is_in_mock_lfs file2.dat \"content 2-3 bin 25 bytes\" &&\n+\t\ttest_file_is_in_mock_lfs \"path with spaces/file3.bin\" \"content 2-3 bin 25 bytes\" &&\n+\t\ttest_file_is_in_mock_lfs file4.bin \"content 4 bin 26 bytes XX\" &&\n+\n+\t\ttest_file_count_in_dir \".git/mock-storage/local\" 2 &&\n+\t\ttest_file_count_in_dir \".git/mock-storage/remote\" 2\n+\t)\n+'\n+\n+test_expect_success 'Store files in Mock LFS based on extension (dat)' '\n+\tclient_view \"//depot/... //client/...\" &&\n+\ttest_when_finished cleanup_git &&\n+\t(\n+\t\tcd \"$git\" &&\n+\t\tgit init . &&\n+\t\tgit config git-p4.useClientSpec true &&\n+\t\tgit config git-p4.largeFileSystem MockLFS &&\n+\t\tgit config git-p4.largeFileExtensions dat &&\n+\t\tgit config git-p4.pushLargeFiles True &&\n+\t\tgit p4 clone --destination=\"$git\" //depot@all &&\n+\n+\t\ttest_file_is_not_in_mock_lfs file1.txt \"content 1 txt 23 bytes\" &&\n+\t\ttest_file_is_in_mock_lfs file2.dat \"content 2-3 bin 25 bytes\" &&\n+\t\ttest_file_is_not_in_mock_lfs \"path with spaces/file3.bin\" \"content 2-3 bin 25 bytes\" &&\n+\t\ttest_file_is_not_in_mock_lfs file4.bin \"content 4 bin 26 bytes XX\" &&\n+\n+\t\ttest_file_count_in_dir \".git/mock-storage/local\" 1 &&\n+\t\ttest_file_count_in_dir \".git/mock-storage/remote\" 1\n+\t)\n+'\n+\n+test_expect_success 'Remove file from repo and store files in Mock LFS based on size (>24 bytes)' '\n+\tclient_view \"//depot/... //client/...\" &&\n+\t(\n+\t\tcd \"$cli\" &&\n+\t\tp4 delete file4.bin &&\n+\t\tp4 submit -d \"Remove file\"\n+\t) &&\n+\n+\tclient_view \"//depot/... //client/...\" &&\n+\ttest_when_finished cleanup_git &&\n+\t(\n+\t\tcd \"$git\" &&\n+\t\tgit init . &&\n+\t\tgit config git-p4.useClientSpec true &&\n+\t\tgit config git-p4.largeFileSystem MockLFS &&\n+\t\tgit config git-p4.largeFileThreshold 24 &&\n+\t\tgit config git-p4.pushLargeFiles True &&\n+\t\tgit p4 clone --destination=\"$git\" //depot@all &&\n+\n+\t\ttest_file_is_not_in_mock_lfs file1.txt \"content 1 txt 23 bytes\" &&\n+\t\ttest_file_is_in_mock_lfs file2.dat \"content 2-3 bin 25 bytes\" &&\n+\t\ttest_file_is_in_mock_lfs \"path with spaces/file3.bin\" \"content 2-3 bin 25 bytes\" &&\n+\t\ttest_file_is_deleted_in_mock_lfs file4.bin \"content 4 bin 26 bytes XX\" &&\n+\n+\t\ttest_file_count_in_dir \".git/mock-storage/local\" 2 &&\n+\t\ttest_file_count_in_dir \".git/mock-storage/remote\" 2\n+\t)\n+'\n+\n+test_expect_success 'Run git p4 submit in repo configured with large file system' '\n+\tclient_view \"//depot/... //client/...\" &&\n+\ttest_when_finished cleanup_git &&\n+\t(\n+\t\tcd \"$git\" &&\n+\t\tgit init . &&\n+\t\tgit config git-p4.useClientSpec true &&\n+\t\tgit config git-p4.largeFileSystem MockLFS &&\n+\t\tgit config git-p4.largeFileThreshold 24 &&\n+\t\tgit config git-p4.pushLargeFiles True &&\n+\t\tgit p4 clone --destination=\"$git\" //depot@all &&\n+\n+\t\ttest_must_fail git p4 submit\n+\t)\n+'\n+\n+test_expect_success 'kill p4d' '\n+\tkill_p4d\n+'\n+\n+test_done\n-- \n2.5.1\n"},{"id":"270450","messageId":"1442875273-48610-8-git-send-email-larsxschneider@gmail.com","threadId":"40399","inReplyTo":"1442875273-48610-1-git-send-email-larsxschneider@gmail.com","subject":"[PATCH v7 7/7] git-p4: add Git LFS backend for large file system","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-09-21T22:41:13Z","receivedAt":"2015-09-21T22:41:13Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider \n\nAdd example implementation including test cases for the large file\nsystem using Git LFS.\n\nPushing files to the Git LFS server is not tested.\n\nSigned-off-by: Lars Schneider \n---\n git-p4.py | 72 ++++++++++++\n t/t9824-git-p4-git-lfs.sh | 288 ++++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 360 insertions(+)\n create mode 100755 t/t9824-git-p4-git-lfs.sh\n\ndiff --git a/git-p4.py b/git-p4.py\nindex 7a80ebd..fce1d65 100755\n--- a/git-p4.py\n+++ b/git-p4.py\n@@ -1038,6 +1038,78 @@ class MockLFS(LargeFileSystem):\n os.makedirs(remotePath)\n shutil.copyfile(localLargeFile, os.path.join(remotePath, os.path.basename(localLargeFile)))\n \n+class GitLFS(LargeFileSystem):\n+ \"\"\"Git LFS as backend for the git-p4 large file system.\n+ See https://git-lfs.github.com/ for details.\"\"\"\n+\n+ def __init__(self, *args):\n+ LargeFileSystem.__init__(self, *args)\n+ self.baseGitAttributes = []\n+\n+ def generatePointer(self, cloneDestination, contentFile):\n+ \"\"\"Generate a Git LFS pointer for the content. Return LFS Pointer file\n+ mode and content which is stored in the Git repository instead of\n+ the actual content. Return also the new location of the actual\n+ content.\n+ \"\"\"\n+ pointerProcess = subprocess.Popen(\n+ ['git', 'lfs', 'pointer', '--file=' + contentFile],\n+ stdout=subprocess.PIPE\n+ )\n+ pointerFile = pointerProcess.stdout.read()\n+ if pointerProcess.wait():\n+ os.remove(contentFile)\n+ die('git-lfs pointer command failed. Did you install the extension?')\n+ pointerContents = [i+'\\n' for i in pointerFile.split('\\n')[2:][:-1]]\n+ oid = pointerContents[1].split(' ')[1].split(':')[1][:-1]\n+ localLargeFile = os.path.join(\n+ cloneDestination,\n+ '.git', 'lfs', 'objects', oid[:2], oid[2:4],\n+ oid,\n+ )\n+ # LFS Spec states that pointer files should not have the executable bit set.\n+ gitMode = '100644'\n+ return (gitMode, pointerContents, localLargeFile)\n+\n+ def pushFile(self, localLargeFile):\n+ uploadProcess = subprocess.Popen(\n+ ['git', 'lfs', 'push', '--object-id', 'origin', os.path.basename(localLargeFile)]\n+ )\n+ if uploadProcess.wait():\n+ die('git-lfs push command failed. Did you define a remote?')\n+\n+ def generateGitAttributes(self):\n+ return (\n+ self.baseGitAttributes +\n+ [\n+ '\\n',\n+ '#\\n',\n+ '# Git LFS (see https://git-lfs.github.com/)\\n',\n+ '#\\n',\n+ ] +\n+ ['*.' + f.replace(' ', '[[:space:]]') + ' filter=lfs -text\\n'\n+ for f in sorted(gitConfigList('git-p4.largeFileExtensions'))\n+ ] +\n+ ['/' + f.replace(' ', '[[:space:]]') + ' filter=lfs -text\\n'\n+ for f in sorted(self.largeFiles) if not self.hasLargeFileExtension(f)\n+ ]\n+ )\n+\n+ def addLargeFile(self, relPath):\n+ LargeFileSystem.addLargeFile(self, relPath)\n+ self.writeToGitStream('100644', '.gitattributes', self.generateGitAttributes())\n+\n+ def removeLargeFile(self, relPath):\n+ LargeFileSystem.removeLargeFile(self, relPath)\n+ self.writeToGitStream('100644', '.gitattributes', self.generateGitAttributes())\n+\n+ def processContent(self, cloneDestination, git_mode, relPath, contents):\n+ if relPath == '.gitattributes':\n+ self.baseGitAttributes = contents\n+ return (git_mode, self.generateGitAttributes())\n+ else:\n+ return LargeFileSystem.processContent(self, cloneDestination, git_mode, relPath, contents)\n+\n class Command:\n def __init__(self):\n self.usage = \"usage: %prog [options]\"\ndiff --git a/t/t9824-git-p4-git-lfs.sh b/t/t9824-git-p4-git-lfs.sh\nnew file mode 100755\nindex 0000000..7385a2b\n--- /dev/null\n+++ b/t/t9824-git-p4-git-lfs.sh\n@@ -0,0 +1,288 @@\n+#!/bin/sh\n+\n+test_description='Clone repositories and store files in Git LFS'\n+\n+. ./lib-git-p4.sh\n+\n+git lfs help >/dev/null 2>&1 || {\n+\tskip_all='skipping git p4 Git LFS tests; Git LFS not found'\n+\ttest_done\n+}\n+\n+test_file_in_lfs () {\n+\tFILE=\"$1\" &&\n+\tSIZE=\"$2\" &&\n+\tEXPECTED_CONTENT=\"$3\" &&\n+\tcat \"$FILE\" | grep \"size $SIZE\" &&\n+\tHASH=$(cat \"$FILE\" | grep \"oid sha256:\" | sed -e \"s/oid sha256://g\") &&\n+\tLFS_FILE=\".git/lfs/objects/$(echo \"$HASH\" | cut -c1-2)/$(echo \"$HASH\" | cut -c3-4)/$HASH\" &&\n+\techo $EXPECTED_CONTENT >expect &&\n+\ttest_path_is_file \"$FILE\" &&\n+\ttest_path_is_file \"$LFS_FILE\" &&\n+\ttest_cmp expect \"$LFS_FILE\"\n+}\n+\n+test_file_count_in_dir () {\n+\tDIR=\"$1\" &&\n+\tEXPECTED_COUNT=\"$2\" &&\n+\tfind \"$DIR\" -type f >actual &&\n+\ttest_line_count = $EXPECTED_COUNT actual\n+}\n+\n+test_expect_success 'start p4d' '\n+\tstart_p4d\n+'\n+\n+test_expect_success 'Create repo with binary files' '\n+\tclient_view \"//depot/... //client/...\" &&\n+\t(\n+\t\tcd \"$cli\" &&\n+\n+\t\techo \"content 1 txt 23 bytes\" >file1.txt &&\n+\t\tp4 add file1.txt &&\n+\t\techo \"content 2-3 bin 25 bytes\" >file2.dat &&\n+\t\tp4 add file2.dat &&\n+\t\tp4 submit -d \"Add text and binary file\" &&\n+\n+\t\tmkdir \"path with spaces\" &&\n+\t\techo \"content 2-3 bin 25 bytes\" >\"path with spaces/file3.bin\" &&\n+\t\tp4 add \"path with spaces/file3.bin\" &&\n+\t\tp4 submit -d \"Add another binary file with same content and spaces in path\" &&\n+\n+\t\techo \"content 4 bin 26 bytes XX\" >file4.bin &&\n+\t\tp4 add file4.bin &&\n+\t\tp4 submit -d \"Add another binary file with different content\"\n+\t)\n+'\n+\n+test_expect_success 'Store files in LFS based on size (>24 bytes)' '\n+\tclient_view \"//depot/... //client/...\" &&\n+\ttest_when_finished cleanup_git &&\n+\t(\n+\t\tcd \"$git\" &&\n+\t\tgit init . &&\n+\t\tgit config git-p4.useClientSpec true &&\n+\t\tgit config git-p4.largeFileSystem GitLFS &&\n+\t\tgit config git-p4.largeFileThreshold 24 &&\n+\t\tgit p4 clone --destination=\"$git\" //depot@all &&\n+\n+\t\ttest_file_in_lfs file2.dat 25 \"content 2-3 bin 25 bytes\" &&\n+\t\ttest_file_in_lfs \"path with spaces/file3.bin\" 25 \"content 2-3 bin 25 bytes\" &&\n+\t\ttest_file_in_lfs file4.bin 26 \"content 4 bin 26 bytes XX\" &&\n+\n+\t\ttest_file_count_in_dir \".git/lfs/objects\" 2 &&\n+\n+\t\tcat >expect <<-\\EOF &&\n+\n+\t\t#\n+\t\t# Git LFS (see https://git-lfs.github.com/)\n+\t\t#\n+\t\t/file2.dat filter=lfs -text\n+\t\t/file4.bin filter=lfs -text\n+\t\t/path[[:space:]]with[[:space:]]spaces/file3.bin filter=lfs -text\n+\t\tEOF\n+\t\ttest_path_is_file .gitattributes &&\n+\t\ttest_cmp expect .gitattributes\n+\t)\n+'\n+\n+test_expect_success 'Store files in LFS based on size (>25 bytes)' '\n+\tclient_view \"//depot/... //client/...\" &&\n+\ttest_when_finished cleanup_git &&\n+\t(\n+\t\tcd \"$git\" &&\n+\t\tgit init . &&\n+\t\tgit config git-p4.useClientSpec true &&\n+\t\tgit config git-p4.largeFileSystem GitLFS &&\n+\t\tgit config git-p4.largeFileThreshold 25 &&\n+\t\tgit p4 clone --destination=\"$git\" //depot@all &&\n+\n+\t\ttest_file_in_lfs file4.bin 26 \"content 4 bin 26 bytes XX\" &&\n+\t\ttest_file_count_in_dir \".git/lfs/objects\" 1 &&\n+\n+\t\tcat >expect <<-\\EOF &&\n+\n+\t\t#\n+\t\t# Git LFS (see https://git-lfs.github.com/)\n+\t\t#\n+\t\t/file4.bin filter=lfs -text\n+\t\tEOF\n+\t\ttest_path_is_file .gitattributes &&\n+\t\ttest_cmp expect .gitattributes\n+\t)\n+'\n+\n+test_expect_success 'Store files in LFS based on extension (dat)' '\n+\tclient_view \"//depot/... //client/...\" &&\n+\ttest_when_finished cleanup_git &&\n+\t(\n+\t\tcd \"$git\" &&\n+\t\tgit init . &&\n+\t\tgit config git-p4.useClientSpec true &&\n+\t\tgit config git-p4.largeFileSystem GitLFS &&\n+\t\tgit config git-p4.largeFileExtensions dat &&\n+\t\tgit p4 clone --destination=\"$git\" //depot@all &&\n+\n+\t\ttest_file_in_lfs file2.dat 25 \"content 2-3 bin 25 bytes\" &&\n+\t\ttest_file_count_in_dir \".git/lfs/objects\" 1 &&\n+\n+\t\tcat >expect <<-\\EOF &&\n+\n+\t\t#\n+\t\t# Git LFS (see https://git-lfs.github.com/)\n+\t\t#\n+\t\t*.dat filter=lfs -text\n+\t\tEOF\n+\t\ttest_path_is_file .gitattributes &&\n+\t\ttest_cmp expect .gitattributes\n+\t)\n+'\n+\n+test_expect_success 'Store files in LFS based on size (>25 bytes) and extension (dat)' '\n+\tclient_view \"//depot/... //client/...\" &&\n+\ttest_when_finished cleanup_git &&\n+\t(\n+\t\tcd \"$git\" &&\n+\t\tgit init . &&\n+\t\tgit config git-p4.useClientSpec true &&\n+\t\tgit config git-p4.largeFileSystem GitLFS &&\n+\t\tgit config git-p4.largeFileExtensions dat &&\n+\t\tgit config git-p4.largeFileThreshold 25 &&\n+\t\tgit p4 clone --destination=\"$git\" //depot@all &&\n+\n+\t\ttest_file_in_lfs file2.dat 25 \"content 2-3 bin 25 bytes\" &&\n+\t\ttest_file_in_lfs file4.bin 26 \"content 4 bin 26 bytes XX\" &&\n+\t\ttest_file_count_in_dir \".git/lfs/objects\" 2 &&\n+\n+\t\tcat >expect <<-\\EOF &&\n+\n+\t\t#\n+\t\t# Git LFS (see https://git-lfs.github.com/)\n+\t\t#\n+\t\t*.dat filter=lfs -text\n+\t\t/file4.bin filter=lfs -text\n+\t\tEOF\n+\t\ttest_path_is_file .gitattributes &&\n+\t\ttest_cmp expect .gitattributes\n+\t)\n+'\n+\n+test_expect_success 'Remove file from repo and store files in LFS based on size (>24 bytes)' '\n+\tclient_view \"//depot/... //client/...\" &&\n+\t(\n+\t\tcd \"$cli\" &&\n+\t\tp4 delete file4.bin &&\n+\t\tp4 submit -d \"Remove file\"\n+\t) &&\n+\n+\tclient_view \"//depot/... //client/...\" &&\n+\ttest_when_finished cleanup_git &&\n+\t(\n+\t\tcd \"$git\" &&\n+\t\tgit init . &&\n+\t\tgit config git-p4.useClientSpec true &&\n+\t\tgit config git-p4.largeFileSystem GitLFS &&\n+\t\tgit config git-p4.largeFileThreshold 24 &&\n+\t\tgit p4 clone --destination=\"$git\" //depot@all &&\n+\n+\t\ttest_file_in_lfs file2.dat 25 \"content 2-3 bin 25 bytes\" &&\n+\t\ttest_file_in_lfs \"path with spaces/file3.bin\" 25 \"content 2-3 bin 25 bytes\" &&\n+\t\t! test_path_is_file file4.bin &&\n+\t\ttest_file_count_in_dir \".git/lfs/objects\" 2 &&\n+\n+\t\tcat >expect <<-\\EOF &&\n+\n+\t\t#\n+\t\t# Git LFS (see https://git-lfs.github.com/)\n+\t\t#\n+\t\t/file2.dat filter=lfs -text\n+\t\t/path[[:space:]]with[[:space:]]spaces/file3.bin filter=lfs -text\n+\t\tEOF\n+\t\ttest_path_is_file .gitattributes &&\n+\t\ttest_cmp expect .gitattributes\n+\t)\n+'\n+\n+test_expect_success 'Add .gitattributes and store files in LFS based on size (>24 bytes)' '\n+\tclient_view \"//depot/... //client/...\" &&\n+\t(\n+\t\tcd \"$cli\" &&\n+\t\techo \"*.txt text\" >.gitattributes &&\n+\t\tp4 add .gitattributes &&\n+\t\tp4 submit -d \"Add .gitattributes\"\n+\t) &&\n+\n+\tclient_view \"//depot/... //client/...\" &&\n+\ttest_when_finished cleanup_git &&\n+\t(\n+\t\tcd \"$git\" &&\n+\t\tgit init . &&\n+\t\tgit config git-p4.useClientSpec true &&\n+\t\tgit config git-p4.largeFileSystem GitLFS &&\n+\t\tgit config git-p4.largeFileThreshold 24 &&\n+\t\tgit p4 clone --destination=\"$git\" //depot@all &&\n+\n+\t\ttest_file_in_lfs file2.dat 25 \"content 2-3 bin 25 bytes\" &&\n+\t\ttest_file_in_lfs \"path with spaces/file3.bin\" 25 \"content 2-3 bin 25 bytes\" &&\n+\t\t! test_path_is_file file4.bin &&\n+\t\ttest_file_count_in_dir \".git/lfs/objects\" 2 &&\n+\n+\t\tcat >expect <<-\\EOF &&\n+\t\t*.txt text\n+\n+\t\t#\n+\t\t# Git LFS (see https://git-lfs.github.com/)\n+\t\t#\n+\t\t/file2.dat filter=lfs -text\n+\t\t/path[[:space:]]with[[:space:]]spaces/file3.bin filter=lfs -text\n+\t\tEOF\n+\t\ttest_path_is_file .gitattributes &&\n+\t\ttest_cmp expect .gitattributes\n+\t)\n+'\n+\n+test_expect_success 'Add big files to repo and store files in LFS based on compressed size (>28 bytes)' '\n+\tclient_view \"//depot/... //client/...\" &&\n+\t(\n+\t\tcd \"$cli\" &&\n+\t\techo \"content 5 bin 40 bytes XXXXXXXXXXXXXXXX\" >file5.bin &&\n+\t\tp4 add file5.bin &&\n+\t\tp4 submit -d \"Add file with small footprint after compression\" &&\n+\n+\t\techo \"content 6 bin 39 bytes XXXXXYYYYYZZZZZ\" >file6.bin &&\n+\t\tp4 add file6.bin &&\n+\t\tp4 submit -d \"Add file with large footprint after compression\"\n+\t) &&\n+\n+\tclient_view \"//depot/... //client/...\" &&\n+\ttest_when_finished cleanup_git &&\n+\t(\n+\t\tcd \"$git\" &&\n+\t\tgit init . &&\n+\t\tgit config git-p4.useClientSpec true &&\n+\t\tgit config git-p4.largeFileSystem GitLFS &&\n+\t\tgit config git-p4.largeFileCompressedThreshold 28 &&\n+\t\t# We only import HEAD here (\"@all\" is missing!)\n+\t\tgit p4 clone --destination=\"$git\" //depot &&\n+\n+\t\ttest_file_in_lfs file6.bin 13 \"content 6 bin 39 bytes XXXXXYYYYYZZZZZ\"\n+\t\ttest_file_count_in_dir \".git/lfs/objects\" 1 &&\n+\n+\t\tcat >expect <<-\\EOF &&\n+\t\t*.txt text\n+\n+\t\t#\n+\t\t# Git LFS (see https://git-lfs.github.com/)\n+\t\t#\n+\t\t/file6.bin filter=lfs -text\n+\t\tEOF\n+\t\ttest_path_is_file .gitattributes &&\n+\t\ttest_cmp expect .gitattributes\n+\t)\n+'\n+\n+test_expect_success 'kill p4d' '\n+\tkill_p4d\n+'\n+\n+test_done\n-- \n2.5.1\n"},{"id":"270535","messageId":"xmqqd1xa15ws.fsf@gitster.mtv.corp.google.com","threadId":"40399","inReplyTo":"1442875273-48610-2-git-send-email-larsxschneider@gmail.com","subject":"Re: [PATCH v7 1/7] git-p4: add optional type specifier to gitConfig reader","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-09-22T21:49:07Z","receivedAt":"2015-09-22T21:49:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"larsxschneider@gmail.com writes:\n\n> From: Lars Schneider \n>\n> The functions “gitConfig” and “gitConfigBool” are almost identical. Make “gitConfig” more generic by adding an optional type specifier. Use the type specifier “—bool” with “gitConfig” to implement “gitConfigBool. This prepares the implementation of other type specifiers such as “—int”.\n\nWhat is this overlong single line paragraph? Is this a MUA artifact\non my end?\n"},{"id":"270537","messageId":"xmqq8u7y1580.fsf@gitster.mtv.corp.google.com","threadId":"40399","inReplyTo":"1442875273-48610-1-git-send-email-larsxschneider@gmail.com","subject":"Re: [PATCH v7 0/7] git-p4: add support for large file systems","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-09-22T22:03:59Z","receivedAt":"2015-09-22T22:03:59Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"larsxschneider@gmail.com writes:\n\n> From: Lars Schneider \n>\n> diff to v6:\n> * fix spaces in .gitattributes for Git-LFS files (old \"[:space:]\", new \"[[:space:]]\")\n> * generated patch on top of next (95c4325) to resolve merge conflicts\n>\n> I am sorry about the \"[:space:]\" bug this late. I had the fix in my development\n> branch but missed to apply it in this patch series.\n>\n> @Junio:\n> Your conflict resolution in pu looks good. Nevertheless, I made this patch on\n> top of next to ease the integration. I hope this is ok.\n\nPlease don't rebase without a good reason, especially after you\nchecked the conflict resolution is OK and your reroll does not\naffect the way the conflicts are resolved. If you based your\nv6 patch on v2.6.0-rc0 and then your v7 patch needs to use something\nthat did not exist v2.6.0-rc3, you cannot avoid rebasing on top of\nthat newer codebase to use that new feature, but otherwise, no.\n\nThis is because I prefer to apply the new series to the same base\nversion so that each step can be compared with the corresponding\nstep in the previous round.\n\nI even try hard to keep the commits from the older round if the\npatch text and log message are unchanged. This time, I had to\nbackport [v7 6/7] to apply to the same base before noticing and\nverifying that [v7 7/7] is the only thing that was changed in this\nround. All the other ones turned out to be identical.\n\nHence, the end result for me was\n\n $ git checkout ls/p4-lfs\n $ git reset --hard HEAD^\n $ git am -s git-p4-lfs-7-of-7.mbox\n\nbut it took me a lot longer than necessary because of the rebase.\n\nThanks.\n"},{"id":"270564","messageId":"C363CF9F-0104-4874-9C88-F966CFD2EA77@gmail.com","threadId":"40399","inReplyTo":"xmqqd1xa15ws.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v7 1/7] git-p4: add optional type specifier to gitConfig reader","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-09-23T07:38:39Z","receivedAt":"2015-09-23T07:38:39Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\nOn 22 Sep 2015, at 23:49, Junio C Hamano wrote:\n\n> larsxschneider@gmail.com writes:\n> \n>> From: Lars Schneider \n>> \n>> The functions “gitConfig” and “gitConfigBool” are almost identical. Make “gitConfig” more generic by adding an optional type specifier. Use the type specifier “—bool” with “gitConfig” to implement “gitConfigBool. This prepares the implementation of other type specifiers such as “—int”.\n> \n> What is this overlong single line paragraph? Is this a MUA artifact\n> on my end?\n> \nNo. It looks like I messed that up. Same is true for the commit “git-p4: add gitConfigInt reader”. I’ll be more careful in the future.\n\nSorry,\nLars\n"},{"id":"270565","messageId":"84A300C5-3AE7-46E7-A95E-9C19F20431EC@gmail.com","threadId":"40399","inReplyTo":"xmqq8u7y1580.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v7 0/7] git-p4: add support for large file systems","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-09-23T07:41:34Z","receivedAt":"2015-09-23T07:41:34Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\nOn 23 Sep 2015, at 00:03, Junio C Hamano wrote:\n\n> larsxschneider@gmail.com writes:\n> \n>> From: Lars Schneider \n>> \n>> diff to v6:\n>> * fix spaces in .gitattributes for Git-LFS files (old \"[:space:]\", new \"[[:space:]]\")\n>> * generated patch on top of next (95c4325) to resolve merge conflicts\n>> \n>> I am sorry about the \"[:space:]\" bug this late. I had the fix in my development\n>> branch but missed to apply it in this patch series.\n>> \n>> @Junio:\n>> Your conflict resolution in pu looks good. Nevertheless, I made this patch on\n>> top of next to ease the integration. I hope this is ok.\n> \n> Please don't rebase without a good reason, especially after you\n> checked the conflict resolution is OK and your reroll does not\n> affect the way the conflicts are resolved. If you based your\n> v6 patch on v2.6.0-rc0 and then your v7 patch needs to use something\n> that did not exist v2.6.0-rc3, you cannot avoid rebasing on top of\n> that newer codebase to use that new feature, but otherwise, no.\n> \n> This is because I prefer to apply the new series to the same base\n> version so that each step can be compared with the corresponding\n> step in the previous round.\n> \n> I even try hard to keep the commits from the older round if the\n> patch text and log message are unchanged. This time, I had to\n> backport [v7 6/7] to apply to the same base before noticing and\n> verifying that [v7 7/7] is the only thing that was changed in this\n> round. All the other ones turned out to be identical.\n> \n> Hence, the end result for me was\n> \n> $ git checkout ls/p4-lfs\n> $ git reset --hard HEAD^\n> $ git am -s git-p4-lfs-7-of-7.mbox\n> \n> but it took me a lot longer than necessary because of the rebase.\n> \nUnderstood. Thanks for taking the time and explaining me your workflow. That helps a lot as my intention was to save your time, not waste it.\n\nSorry,\nLars\n"},{"id":"270566","messageId":"CAE5ih7_PBRMgobhBATUMOaRH8yPWWnEVDiFVdk+qjRzoyPq+CQ@mail.gmail.com","threadId":"40399","inReplyTo":"1442875273-48610-1-git-send-email-larsxschneider@gmail.com","subject":"Re: [PATCH v7 0/7] git-p4: add support for large file systems","fromName":"Luke Diamand","fromEmail":"luke@diamand.org","sentAt":"2015-09-23T07:58:26Z","receivedAt":"2015-09-23T07:58:26Z","isPatch":true,"sender":{"key":"luke@diamand.org","avatar":"https://avatars.githubusercontent.com/u/5330967?v=4"},"body":"On 21 September 2015 at 23:41, wrote:\n> From: Lars Schneider \n>\n> diff to v6:\n> * fix spaces in .gitattributes for Git-LFS files (old \"[:space:]\", new \"[[:space:]]\")\n> * generated patch on top of next (95c4325) to resolve merge conflicts\n\nTrying this out by hand (rather than using the test cases) it crashes\nfor me. Am I doing something wrong?\n\n- I've got the version of git-p4.py from\nef93b8df71f6d8a7936786c989714868c2d3540c in Junio's ls/p4-lfs branch.\n- I have git-lfs v0.60 (just downloaded the binary distribution).\n\nThen I created a p4 repo, and added two files in Perforce, small.txt\nand bigfile. bigfile is 400k.\n\nDoing a regular git clone works fine:\n\n$ mkdir depot\n$ cd depot\n$ git init .\n$ git p4 sync //depot\n$ git checkout p4/master\n\n(I'm using git init + git sync rather than git clone so that I can\nsetup the git config variables I need in the next step).\n\nDoing a clone with LFS support though doesn't work:\n\n$ mkdir depot2\n$ cd depot2\n$ git init\n$ git config git-p4.largeFileSystem GitLFS\n$ git config git-p4.largeFileThreshold 100k\n$ git p4 sync //depot\nDoing initial import of //depot/ from revision #head into refs/remotes/p4/master\nTraceback (most recent call last):\n File \"/home/ldiamand/git/git/git-p4.py\", line 3624, in \n main()\n File \"/home/ldiamand/git/git/git-p4.py\", line 3618, in main\n if not cmd.run(args):\n File \"/home/ldiamand/git/git/git-p4.py\", line 3298, in run\n self.importHeadRevision(revision)\n File \"/home/ldiamand/git/git/git-p4.py\", line 3085, in importHeadRevision\n self.commit(details, self.extractFilesFromCommit(details), self.branch)\n File \"/home/ldiamand/git/git/git-p4.py\", line 2651, in commit\n self.streamP4Files(new_files)\n File \"/home/ldiamand/git/git/git-p4.py\", line 2565, in streamP4Files\n cb=streamP4FilesCbSelf)\n File \"/home/ldiamand/git/git/git-p4.py\", line 501, in p4CmdList\n cb(entry)\n File \"/home/ldiamand/git/git/git-p4.py\", line 2559, in streamP4FilesCbSelf\n self.streamP4FilesCb(entry)\n File \"/home/ldiamand/git/git/git-p4.py\", line 2501, in streamP4FilesCb\n self.streamOneP4File(self.stream_file, self.stream_contents)\n File \"/home/ldiamand/git/git/git-p4.py\", line 2451, in streamOneP4File\n (git_mode, contents) =\nself.largeFileSystem.processContent(self.cloneDestination, git_mode,\nrelPath, contents)\n File \"/home/ldiamand/git/git/git-p4.py\", line 1111, in processContent\n return LargeFileSystem.processContent(self, cloneDestination,\ngit_mode, relPath, contents)\n File \"/home/ldiamand/git/git/git-p4.py\", line 1004, in processContent\n (git_mode, contents, localLargeFile) =\nself.generatePointer(cloneDestination, contentTempFile)\n File \"/home/ldiamand/git/git/git-p4.py\", line 1068, in generatePointer\n oid,\n File \"/usr/lib/python2.7/posixpath.py\", line 70, in join\n elif path == '' or path.endswith('/'):\nAttributeError: 'NoneType' object has no attribute 'endswith'\n\n\nThe test cases work fine, t9823 and t9824.\n\nThanks\nLuke\n"},{"id":"270567","messageId":"CE59FE83-4EE1-41EF-85B6-63EA51BD9E3B@gmail.com","threadId":"40399","inReplyTo":"CAE5ih7_PBRMgobhBATUMOaRH8yPWWnEVDiFVdk+qjRzoyPq+CQ@mail.gmail.com","subject":"Re: [PATCH v7 0/7] git-p4: add support for large file systems","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-09-23T08:18:52Z","receivedAt":"2015-09-23T08:18:52Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\nOn 23 Sep 2015, at 09:58, Luke Diamand wrote:\n\n> On 21 September 2015 at 23:41, wrote:\n>> From: Lars Schneider \n>> \n>> diff to v6:\n>> * fix spaces in .gitattributes for Git-LFS files (old \"[:space:]\", new \"[[:space:]]\")\n>> * generated patch on top of next (95c4325) to resolve merge conflicts\n> \n> Trying this out by hand (rather than using the test cases) it crashes\n> for me. Am I doing something wrong?\n> \n> - I've got the version of git-p4.py from\n> ef93b8df71f6d8a7936786c989714868c2d3540c in Junio's ls/p4-lfs branch.\n> - I have git-lfs v0.60 (just downloaded the binary distribution).\n> \n> Then I created a p4 repo, and added two files in Perforce, small.txt\n> and bigfile. bigfile is 400k.\n> \n> Doing a regular git clone works fine:\n> \n> $ mkdir depot\n> $ cd depot\n> $ git init .\n> $ git p4 sync //depot\n> $ git checkout p4/master\n> \n> (I'm using git init + git sync rather than git clone so that I can\n> setup the git config variables I need in the next step).\n> \n> Doing a clone with LFS support though doesn't work:\n> \n> $ mkdir depot2\n> $ cd depot2\n> $ git init\n> $ git config git-p4.largeFileSystem GitLFS\n> $ git config git-p4.largeFileThreshold 100k\n> $ git p4 sync //depot\n> Doing initial import of //depot/ from revision #head into refs/remotes/p4/master\n> Traceback (most recent call last):\n> File \"/home/ldiamand/git/git/git-p4.py\", line 3624, in \n> main()\n> File \"/home/ldiamand/git/git/git-p4.py\", line 3618, in main\n> if not cmd.run(args):\n> File \"/home/ldiamand/git/git/git-p4.py\", line 3298, in run\n> self.importHeadRevision(revision)\n> File \"/home/ldiamand/git/git/git-p4.py\", line 3085, in importHeadRevision\n> self.commit(details, self.extractFilesFromCommit(details), self.branch)\n> File \"/home/ldiamand/git/git/git-p4.py\", line 2651, in commit\n> self.streamP4Files(new_files)\n> File \"/home/ldiamand/git/git/git-p4.py\", line 2565, in streamP4Files\n> cb=streamP4FilesCbSelf)\n> File \"/home/ldiamand/git/git/git-p4.py\", line 501, in p4CmdList\n> cb(entry)\n> File \"/home/ldiamand/git/git/git-p4.py\", line 2559, in streamP4FilesCbSelf\n> self.streamP4FilesCb(entry)\n> File \"/home/ldiamand/git/git/git-p4.py\", line 2501, in streamP4FilesCb\n> self.streamOneP4File(self.stream_file, self.stream_contents)\n> File \"/home/ldiamand/git/git/git-p4.py\", line 2451, in streamOneP4File\n> (git_mode, contents) =\n> self.largeFileSystem.processContent(self.cloneDestination, git_mode,\n> relPath, contents)\n> File \"/home/ldiamand/git/git/git-p4.py\", line 1111, in processContent\n> return LargeFileSystem.processContent(self, cloneDestination,\n> git_mode, relPath, contents)\n> File \"/home/ldiamand/git/git/git-p4.py\", line 1004, in processContent\n> (git_mode, contents, localLargeFile) =\n> self.generatePointer(cloneDestination, contentTempFile)\n> File \"/home/ldiamand/git/git/git-p4.py\", line 1068, in generatePointer\n> oid,\n> File \"/usr/lib/python2.7/posixpath.py\", line 70, in join\n> elif path == '' or path.endswith('/'):\n> AttributeError: 'NoneType' object has no attribute 'endswith'\n> \nConfirmed. This is a bug!\n\nCan you try the following sequence of commands?\n\n$ mkdir depot2\n$ cd depot2\n$ git init\n$ git config git-p4.largeFileSystem GitLFS\n$ git config git-p4.largeFileThreshold 100k\n$ git p4 clone --destination=. //depot\n\nI’ve always used “clone” and all test cases uses “clone”. I will fix it for “sync”.\n\nThanks for reporting,\nLars\n"},{"id":"270568","messageId":"A29B974F-2B42-475F-92B6-8B25A54FCDEE@gmail.com","threadId":"40399","inReplyTo":"CE59FE83-4EE1-41EF-85B6-63EA51BD9E3B@gmail.com","subject":"Re: [PATCH v7 0/7] git-p4: add support for large file systems","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-09-23T08:50:58Z","receivedAt":"2015-09-23T08:50:58Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\nOn 23 Sep 2015, at 10:18, Lars Schneider wrote:\n\n> \n> On 23 Sep 2015, at 09:58, Luke Diamand wrote:\n> \n>> On 21 September 2015 at 23:41, wrote:\n>>> From: Lars Schneider \n>>> \n>>> diff to v6:\n>>> * fix spaces in .gitattributes for Git-LFS files (old \"[:space:]\", new \"[[:space:]]\")\n>>> * generated patch on top of next (95c4325) to resolve merge conflicts\n>> \n>> Trying this out by hand (rather than using the test cases) it crashes\n>> for me. Am I doing something wrong?\n>> \n>> - I've got the version of git-p4.py from\n>> ef93b8df71f6d8a7936786c989714868c2d3540c in Junio's ls/p4-lfs branch.\n>> - I have git-lfs v0.60 (just downloaded the binary distribution).\n>> \n>> Then I created a p4 repo, and added two files in Perforce, small.txt\n>> and bigfile. bigfile is 400k.\n>> \n>> Doing a regular git clone works fine:\n>> \n>> $ mkdir depot\n>> $ cd depot\n>> $ git init .\n>> $ git p4 sync //depot\n>> $ git checkout p4/master\n>> \n>> (I'm using git init + git sync rather than git clone so that I can\n>> setup the git config variables I need in the next step).\n>> \n>> Doing a clone with LFS support though doesn't work:\n>> \n>> $ mkdir depot2\n>> $ cd depot2\n>> $ git init\n>> $ git config git-p4.largeFileSystem GitLFS\n>> $ git config git-p4.largeFileThreshold 100k\n>> $ git p4 sync //depot\n>> Doing initial import of //depot/ from revision #head into refs/remotes/p4/master\n>> Traceback (most recent call last):\n>> File \"/home/ldiamand/git/git/git-p4.py\", line 3624, in \n>> main()\n>> File \"/home/ldiamand/git/git/git-p4.py\", line 3618, in main\n>> if not cmd.run(args):\n>> File \"/home/ldiamand/git/git/git-p4.py\", line 3298, in run\n>> self.importHeadRevision(revision)\n>> File \"/home/ldiamand/git/git/git-p4.py\", line 3085, in importHeadRevision\n>> self.commit(details, self.extractFilesFromCommit(details), self.branch)\n>> File \"/home/ldiamand/git/git/git-p4.py\", line 2651, in commit\n>> self.streamP4Files(new_files)\n>> File \"/home/ldiamand/git/git/git-p4.py\", line 2565, in streamP4Files\n>> cb=streamP4FilesCbSelf)\n>> File \"/home/ldiamand/git/git/git-p4.py\", line 501, in p4CmdList\n>> cb(entry)\n>> File \"/home/ldiamand/git/git/git-p4.py\", line 2559, in streamP4FilesCbSelf\n>> self.streamP4FilesCb(entry)\n>> File \"/home/ldiamand/git/git/git-p4.py\", line 2501, in streamP4FilesCb\n>> self.streamOneP4File(self.stream_file, self.stream_contents)\n>> File \"/home/ldiamand/git/git/git-p4.py\", line 2451, in streamOneP4File\n>> (git_mode, contents) =\n>> self.largeFileSystem.processContent(self.cloneDestination, git_mode,\n>> relPath, contents)\n>> File \"/home/ldiamand/git/git/git-p4.py\", line 1111, in processContent\n>> return LargeFileSystem.processContent(self, cloneDestination,\n>> git_mode, relPath, contents)\n>> File \"/home/ldiamand/git/git/git-p4.py\", line 1004, in processContent\n>> (git_mode, contents, localLargeFile) =\n>> self.generatePointer(cloneDestination, contentTempFile)\n>> File \"/home/ldiamand/git/git/git-p4.py\", line 1068, in generatePointer\n>> oid,\n>> File \"/usr/lib/python2.7/posixpath.py\", line 70, in join\n>> elif path == '' or path.endswith('/'):\n>> AttributeError: 'NoneType' object has no attribute 'endswith'\n>> \n> Confirmed. This is a bug!\n> \n> Can you try the following sequence of commands?\n> \n> $ mkdir depot2\n> $ cd depot2\n> $ git init\n> $ git config git-p4.largeFileSystem GitLFS\n> $ git config git-p4.largeFileThreshold 100k\n> $ git p4 clone --destination=. //depot\n> \n> I’ve always used “clone” and all test cases uses “clone”. I will fix it for “sync”.\n> \n> Thanks for reporting,\n> Lars\n\nI think I found an easy fix. Can you try it?\n\n(in case my mail app messes something up: I changed line 2240 in git-p4.py to 'self.cloneDestination = os.getcwd()’)\n\nThanks,\nLars\n\n\ndiff --git a/git-p4.py b/git-p4.py\nindex 8c7496d..1d1bb87 100755\n--- a/git-p4.py\n+++ b/git-p4.py\n@@ -2237,7 +2237,7 @@ class P4Sync(Command, P4UserMap):\n self.tempBranches = []\n self.tempBranchLocation = \"git-p4-tmp\"\n self.largeFileSystem = None\n- self.cloneDestination = None\n+ self.cloneDestination = os.getcwd()\n\n if gitConfig('git-p4.largeFileSystem'):\n largeFileSystemConstructor = globals()[gitConfig('git-p4.largeFileSystem')]\ndiff --git a/t/t9824-git-p4-git-lfs.sh b/t/t9824-git-p4-git-lfs.sh\nindex 7385a2b..6d374c0 100755\n--- a/t/t9824-git-p4-git-lfs.sh\n+++ b/t/t9824-git-p4-git-lfs.sh\n@@ -86,6 +86,35 @@ test_expect_success 'Store files in LFS based on size (>24 bytes)' '\n )\n '\n\n+test_expect_success 'Store files in LFS based on size (>24 bytes) and use git p4 sync and no client spec' '\n+ test_when_finished cleanup_git &&\n+ (\n+ cd \"$git\" &&\n+ git init . &&\n+ git config git-p4.largeFileSystem GitLFS &&\n+ git config git-p4.largeFileThreshold 24 &&\n+ git p4 sync //depot &&\n+ git checkout p4/master\n+ test_file_in_lfs file2.dat 25 \"content 2-3 bin 25 bytes\" &&\n+ test_file_in_lfs \"path with spaces/file3.bin\" 25 \"content 2-3 bin 25 bytes\" &&\n+ test_file_in_lfs file4.bin 26 \"content 4 bin 26 bytes XX\" &&\n+\n+ test_file_count_in_dir \".git/lfs/objects\" 2 &&\n+\n+ cat >expect <<-\\EOF &&\n+\n+ #\n+ # Git LFS (see https://git-lfs.github.com/)\n+ #\n+ /file2.dat filter=lfs -text\n+ /file4.bin filter=lfs -text\n+ /path[[:space:]]with[[:space:]]spaces/file3.bin filter=lfs -text\n+ EOF\n+ test_path_is_file .gitattributes &&\n+ test_cmp expect .gitattributes\n+ )\n+'\n+\n test_expect_success 'Store files in LFS based on size (>25 bytes)' '\n client_view \"//depot/... //client/...\" &&\n test_when_finished cleanup_git &&\n"},{"id":"270574","messageId":"CAE5ih7_m050wcWZ+7UvL3GscKgSxLWGT+bj4_gSVA8R7RZm=tg@mail.gmail.com","threadId":"40399","inReplyTo":"CAE5ih7_StiXA-c5yER9jhc941vsKfvEUONLXQ4E7JANOhCxMxQ@mail.gmail.com","subject":"Re: [PATCH v7 0/7] git-p4: add support for large file systems","fromName":"Luke Diamand","fromEmail":"luke@diamand.org","sentAt":"2015-09-23T11:25:26Z","receivedAt":"2015-09-23T11:25:26Z","isPatch":true,"sender":{"key":"luke@diamand.org","avatar":"https://avatars.githubusercontent.com/u/5330967?v=4"},"body":"Adding back git@vger.kernel.org, which I inadvertently dropped off the thread.\n\nOn 23 September 2015 at 12:22, Luke Diamand wrote:\n> On 23 September 2015 at 11:09, Lars Schneider wrote:\n>>\n>> On 23 Sep 2015, at 11:22, Luke Diamand wrote:\n>>\n>>> On 23 September 2015 at 09:50, Lars Schneider wrote:\n>>>>\n>>>> On 23 Sep 2015, at 10:18, Lars Schneider wrote:\n>>>\n>>> \n>>>\n>>>>\n>>>> I think I found an easy fix. Can you try it?\n>>>>\n>>>> (in case my mail app messes something up: I changed line 2240 in git-p4.py to 'self.cloneDestination = os.getcwd()’)\n>>>\n>>> It fixes the problem, but in re-running the tests, I'm seeing t9808\n>>> fail which doesn't happen on next.\n>> Confirmed. I fixed it.\n>> Do you think it makes sense to create a new roll v8 for Junio?\n>\n> How about we leave it a day or two in case anything else crawls out of\n> the woodwork?\n>\n> Thanks!\n> Luke\n"},{"id":"270575","messageId":"F05CC31A-ECE2-41E4-9805-F1747DD02713@gmail.com","threadId":"40399","inReplyTo":"CAE5ih7_m050wcWZ+7UvL3GscKgSxLWGT+bj4_gSVA8R7RZm=tg@mail.gmail.com","subject":"Re: [PATCH v7 0/7] git-p4: add support for large file systems","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-09-23T11:42:43Z","receivedAt":"2015-09-23T11:42:43Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\nOn 23 Sep 2015, at 13:25, Luke Diamand wrote:\n\n> Adding back git@vger.kernel.org, which I inadvertently dropped off the thread.\n> \n> On 23 September 2015 at 12:22, Luke Diamand wrote:\n>> On 23 September 2015 at 11:09, Lars Schneider wrote:\n>>> \n>>> On 23 Sep 2015, at 11:22, Luke Diamand wrote:\n>>> \n>>>> On 23 September 2015 at 09:50, Lars Schneider wrote:\n>>>>> \n>>>>> On 23 Sep 2015, at 10:18, Lars Schneider wrote:\n>>>> \n>>>> \n>>>> \n>>>>> \n>>>>> I think I found an easy fix. Can you try it?\n>>>>> \n>>>>> (in case my mail app messes something up: I changed line 2240 in git-p4.py to 'self.cloneDestination = os.getcwd()’)\n>>>> \n>>>> It fixes the problem, but in re-running the tests, I'm seeing t9808\n>>>> fail which doesn't happen on next.\n>>> Confirmed. I fixed it.\n>>> Do you think it makes sense to create a new roll v8 for Junio?\n>> \n>> How about we leave it a day or two in case anything else crawls out of\n>> the woodwork?\n>> \n>> Thanks!\n>> Luke\nsounds good to me!\n\nThanks,\nLars"},{"id":"270647","messageId":"CAE5ih79OBdGP-PgoYWKBn3_xv-w6dfN-3u0iPBDjWbQb_MMbsw@mail.gmail.com","threadId":"40399","inReplyTo":"F05CC31A-ECE2-41E4-9805-F1747DD02713@gmail.com","subject":"Re: [PATCH v7 0/7] git-p4: add support for large file systems","fromName":"Luke Diamand","fromEmail":"luke@diamand.org","sentAt":"2015-09-24T20:44:53Z","receivedAt":"2015-09-24T20:44:53Z","isPatch":true,"sender":{"key":"luke@diamand.org","avatar":"https://avatars.githubusercontent.com/u/5330967?v=4"},"body":"On 23 September 2015 at 12:42, Lars Schneider wrote:\n>\n> On 23 Sep 2015, at 13:25, Luke Diamand wrote:\n>\n>> Adding back git@vger.kernel.org, which I inadvertently dropped off the thread.\n>>\n>> On 23 September 2015 at 12:22, Luke Diamand wrote:\n>>> On 23 September 2015 at 11:09, Lars Schneider wrote:\n>>>>\n>>>> On 23 Sep 2015, at 11:22, Luke Diamand wrote:\n>>>>\n>>>>> On 23 September 2015 at 09:50, Lars Schneider wrote:\n>>>>>>\n>>>>>> On 23 Sep 2015, at 10:18, Lars Schneider wrote:\n>>>>>\n>>>>> \n>>>>>\n>>>>>>\n>>>>>> I think I found an easy fix. Can you try it?\n>>>>>>\n>>>>>> (in case my mail app messes something up: I changed line 2240 in git-p4.py to 'self.cloneDestination = os.getcwd()’)\n>>>>>\n>>>>> It fixes the problem, but in re-running the tests, I'm seeing t9808\n>>>>> fail which doesn't happen on next.\n>>>> Confirmed. I fixed it.\n>>>> Do you think it makes sense to create a new roll v8 for Junio?\n>>>\n>>> How about we leave it a day or two in case anything else crawls out of\n>>> the woodwork?\n>>>\n>>> Thanks!\n>>> Luke\n> sounds good to me!\n\nOK, not sure if I'm just doing something daft here....but it seems to\nbe ignoring the size limit!\n\nI've got the version from the branch:\n\n 8fee565 git-p4: add Git LFS backend for large file system\n\nPlus the couple of oneliner fixes for cloneDestination.\n\nI've created a repo with a file called foo, size 16MB, and another\nfile called foo.mp4, which is just a symlink to foo.\n\nI then do:\n\n$ mkdir a\n$ cd a\n$ git init .\n$ git config git-p4.largeFileSystem GitLFS\n$ git config git-p4.largeFileExtensions mp4\n$ git config git-p4.largeFileThreshold 100k\n$ git-p4.py sync -v //depot\n\nThat reports that both foo and foo.mp4 have been pushed into LFS:\n\n //depot/foo --> foo (16 MB)\n Reading pipe: ['git', 'config', '--bool', 'git-p4.pushLargeFiles']\n foo moved to large file system\n(/home/lgd/git/git/x/git/a/.git/lfs/objects/08/0a/080acf35a507ac9849cfcba47dc2ad83e01b75663a516279c8b9d243b719643e)\n //depot/foo.mp4 --> foo.mp4 (0 MB)\n foo.mp4 moved to large file system\n(/home/lgd/git/git/x/git/a/.git/lfs/objects/2c/26/2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae)\n\nBut the file system says otherwise:\n\n$ ls -l\n-rw-r--r-- 1 lgd lgd 16777216 Sep 24 21:38 foo\n-rw-r--r-- 1 lgd lgd 3 Sep 24 21:38 foo.mp4\n\nAs does git:\n\ngit ls-files --debug\n.gitattributes\n ctime: 1443127106:535552029\n mtime: 1443127106:535552029\n dev: 65025 ino: 13638459\n uid: 1000 gid: 1000\n size: 94 flags: 0\nfoo\n ctime: 1443127106:579552030\n mtime: 1443127106:579552030\n dev: 65025 ino: 13638462\n uid: 1000 gid: 1000\n size: 16777216 flags: 0 <<<<<<<<<<<<< Quite large!\nfoo.mp4\n ctime: 1443127106:599552030\n mtime: 1443127106:599552030\n dev: 65025 ino: 13638463\n uid: 1000 gid: 1000\n size: 3 flags: 0\n\nWhat's going on?\n\nThanks!\nLuke\n"},{"id":"270742","messageId":"571C8128-DFB9-457F-97AE-8F6BC0579465@gmail.com","threadId":"40399","inReplyTo":"CAE5ih79OBdGP-PgoYWKBn3_xv-w6dfN-3u0iPBDjWbQb_MMbsw@mail.gmail.com","subject":"Re: [PATCH v7 0/7] git-p4: add support for large file systems","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-09-25T08:35:09Z","receivedAt":"2015-09-25T08:35:09Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\nOn 24 Sep 2015, at 22:44, Luke Diamand wrote:\n\n> On 23 September 2015 at 12:42, Lars Schneider wrote:\n>> \n>> On 23 Sep 2015, at 13:25, Luke Diamand wrote:\n>> \n>>> Adding back git@vger.kernel.org, which I inadvertently dropped off the thread.\n>>> \n>>> On 23 September 2015 at 12:22, Luke Diamand wrote:\n>>>> On 23 September 2015 at 11:09, Lars Schneider wrote:\n>>>>> \n>>>>> On 23 Sep 2015, at 11:22, Luke Diamand wrote:\n>>>>> \n>>>>>> On 23 September 2015 at 09:50, Lars Schneider wrote:\n>>>>>>> \n>>>>>>> On 23 Sep 2015, at 10:18, Lars Schneider wrote:\n>>>>>> \n>>>>>> \n>>>>>> \n>>>>>>> \n>>>>>>> I think I found an easy fix. Can you try it?\n>>>>>>> \n>>>>>>> (in case my mail app messes something up: I changed line 2240 in git-p4.py to 'self.cloneDestination = os.getcwd()’)\n>>>>>> \n>>>>>> It fixes the problem, but in re-running the tests, I'm seeing t9808\n>>>>>> fail which doesn't happen on next.\n>>>>> Confirmed. I fixed it.\n>>>>> Do you think it makes sense to create a new roll v8 for Junio?\n>>>> \n>>>> How about we leave it a day or two in case anything else crawls out of\n>>>> the woodwork?\n>>>> \n>>>> Thanks!\n>>>> Luke\n>> sounds good to me!\n> \n> OK, not sure if I'm just doing something daft here....but it seems to\n> be ignoring the size limit!\n> \n> I've got the version from the branch:\n> \n> 8fee565 git-p4: add Git LFS backend for large file system\n> \n> Plus the couple of oneliner fixes for cloneDestination.\n> \n> I've created a repo with a file called foo, size 16MB, and another\n> file called foo.mp4, which is just a symlink to foo.\n> \n> I then do:\n> \n> $ mkdir a\n> $ cd a\n> $ git init .\n> $ git config git-p4.largeFileSystem GitLFS\n> $ git config git-p4.largeFileExtensions mp4\n> $ git config git-p4.largeFileThreshold 100k\n> $ git-p4.py sync -v //depot\n> \n> That reports that both foo and foo.mp4 have been pushed into LFS:\n> \n> //depot/foo --> foo (16 MB)\n> Reading pipe: ['git', 'config', '--bool', 'git-p4.pushLargeFiles']\n> foo moved to large file system\n> (/home/lgd/git/git/x/git/a/.git/lfs/objects/08/0a/080acf35a507ac9849cfcba47dc2ad83e01b75663a516279c8b9d243b719643e)\n> //depot/foo.mp4 --> foo.mp4 (0 MB)\n> foo.mp4 moved to large file system\n> (/home/lgd/git/git/x/git/a/.git/lfs/objects/2c/26/2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae)\n> \n> But the file system says otherwise:\n> \n> $ ls -l\n> -rw-r--r-- 1 lgd lgd 16777216 Sep 24 21:38 foo\n> -rw-r--r-- 1 lgd lgd 3 Sep 24 21:38 foo.mp4\n> \n> As does git:\n> \n> git ls-files --debug\n> .gitattributes\n> ctime: 1443127106:535552029\n> mtime: 1443127106:535552029\n> dev: 65025 ino: 13638459\n> uid: 1000 gid: 1000\n> size: 94 flags: 0\n> foo\n> ctime: 1443127106:579552030\n> mtime: 1443127106:579552030\n> dev: 65025 ino: 13638462\n> uid: 1000 gid: 1000\n> size: 16777216 flags: 0 <<<<<<<<<<<<< Quite large!\n> foo.mp4\n> ctime: 1443127106:599552030\n> mtime: 1443127106:599552030\n> dev: 65025 ino: 13638463\n> uid: 1000 gid: 1000\n> size: 3 flags: 0\n> \n> What's going on?\nI believe this is correct. Git-LFS uses the clean/smudge filter to replace the LFS pointer with the actual file on checkout. Therefore you see the actual file!\nYou can find details here: https://github.com/github/git-lfs/blob/master/docs/spec.md#intercepting-git\n\nCan you run `du -hs .git/objects` in your Git repository? This should report a value well below 16 MB.\n\nCheers,\nLars\n"},{"id":"270743","messageId":"56050D1A.2000204@diamand.org","threadId":"40399","inReplyTo":"571C8128-DFB9-457F-97AE-8F6BC0579465@gmail.com","subject":"Re: [PATCH v7 0/7] git-p4: add support for large file systems","fromName":"Luke Diamand","fromEmail":"luke@diamand.org","sentAt":"2015-09-25T09:00:10Z","receivedAt":"2015-09-25T09:00:10Z","isPatch":true,"sender":{"key":"luke@diamand.org","avatar":"https://avatars.githubusercontent.com/u/5330967?v=4"},"body":"On 25/09/15 09:35, Lars Schneider wrote:\n>\n>> size: 3 flags: 0\n>>\n>> What's going on?\n> I believe this is correct. Git-LFS uses the clean/smudge filter to replace the LFS pointer with the actual file on checkout. Therefore you see the actual file!\n> You can find details here: https://github.com/github/git-lfs/blob/master/docs/spec.md#intercepting-git\n>\n> Can you run `du -hs .git/objects` in your Git repository? This should report a value well below 16 MB.\n\nIt does, thanks!\n"},{"id":"270744","messageId":"CAE5ih7890uN-5uP9YkwqGi8AV2P+s=Heban6fvbdJ4DG-WrurA@mail.gmail.com","threadId":"40399","inReplyTo":"1442875273-48610-7-git-send-email-larsxschneider@gmail.com","subject":"Re: [PATCH v7 6/7] git-p4: add support for large file systems","fromName":"Luke Diamand","fromEmail":"luke@diamand.org","sentAt":"2015-09-25T09:09:46Z","receivedAt":"2015-09-25T09:09:46Z","isPatch":true,"sender":{"key":"luke@diamand.org","avatar":"https://avatars.githubusercontent.com/u/5330967?v=4"},"body":"One tiny comment, otherwise this looks good to me.\n\nCan we use test_path_is_missing in place of !test_path_is_file ?\n\n\n\nOn 21 September 2015 at 23:41, wrote:\n> From: Lars Schneider \n>\n> Perforce repositories can contain large (binary) files. Migrating these\n> repositories to Git generates very large local clones. External storage\n> systems such as Git LFS [1], Git Fat [2], Git Media [3], git-annex [4]\n> try to address this problem.\n>\n> Add a generic mechanism to detect large files based on extension,\n> uncompressed size, and/or compressed size.\n>\n> [1] https://git-lfs.github.com/\n> [2] https://github.com/jedbrown/git-fat\n> [3] https://github.com/alebedev/git-media\n> [4] https://git-annex.branchable.com/\n>\n>\n> diff --git a/t/t9823-git-p4-mock-lfs.sh b/t/t9823-git-p4-mock-lfs.sh\n> new file mode 100755\n> index 0000000..5684ee3\n> --- /dev/null\n> +++ b/t/t9823-git-p4-mock-lfs.sh\n> @@ -0,0 +1,170 @@\n> +#!/bin/sh\n> +\n> +test_description='Clone repositories and store files in Mock LFS'\n> +\n> +. ./lib-git-p4.sh\n> +\n> +test_file_is_not_in_mock_lfs () {\n> + FILE=\"$1\" &&\n> + CONTENT=\"$2\" &&\n> + echo \"$CONTENT\" >expect_content &&\n> + test_path_is_file \"$FILE\" &&\n> + test_cmp expect_content \"$FILE\"\n> +}\n> +\n> +test_file_is_in_mock_lfs () {\n> + FILE=\"$1\" &&\n> + CONTENT=\"$2\" &&\n> + LOCAL_STORAGE=\".git/mock-storage/local/$CONTENT\" &&\n> + SERVER_STORAGE=\".git/mock-storage/remote/$CONTENT\" &&\n> + echo \"pointer-$CONTENT\" >expect_pointer &&\n> + echo \"$CONTENT\" >expect_content &&\n> + test_path_is_file \"$FILE\" &&\n> + test_path_is_file \"$LOCAL_STORAGE\" &&\n> + test_path_is_file \"$SERVER_STORAGE\" &&\n> + test_cmp expect_pointer \"$FILE\" &&\n> + test_cmp expect_content \"$LOCAL_STORAGE\" &&\n> + test_cmp expect_content \"$SERVER_STORAGE\"\n> +}\n> +\n> +test_file_is_deleted_in_mock_lfs () {\n> + FILE=\"$1\" &&\n> + CONTENT=\"$2\" &&\n> + LOCAL_STORAGE=\".git/mock-storage/local/$CONTENT\" &&\n> + SERVER_STORAGE=\".git/mock-storage/remote/$CONTENT\" &&\n> + echo \"pointer-$CONTENT\" >expect_pointer &&\n> + echo \"$CONTENT\" >expect_content &&\n> + ! test_path_is_file \"$FILE\" &&\n\nPerhaps use test_path_is_missing instead here?\n\n> + test_path_is_file \"$LOCAL_STORAGE\" &&\n> + test_path_is_file \"$SERVER_STORAGE\" &&\n> + test_cmp expect_content \"$LOCAL_STORAGE\" &&\n> + test_cmp expect_content \"$SERVER_STORAGE\"\n> +}\n> +\n> +test_file_count_in_dir () {\n> + DIR=\"$1\" &&\n> + EXPECTED_COUNT=\"$2\" &&\n> + find \"$DIR\" -type f >actual &&\n> + test_line_count = $EXPECTED_COUNT actual\n> +}\n> +\n> +test_expect_success 'start p4d' '\n> + start_p4d\n> +'\n> +\n> +test_expect_success 'Create repo with binary files' '\n> + client_view \"//depot/... //client/...\" &&\n> + (\n> + cd \"$cli\" &&\n> +\n> + echo \"content 1 txt 23 bytes\" >file1.txt &&\n> + p4 add file1.txt &&\n> + echo \"content 2-3 bin 25 bytes\" >file2.dat &&\n> + p4 add file2.dat &&\n> + p4 submit -d \"Add text and binary file\" &&\n> +\n> + mkdir \"path with spaces\" &&\n> + echo \"content 2-3 bin 25 bytes\" >\"path with spaces/file3.bin\" &&\n> + p4 add \"path with spaces/file3.bin\" &&\n> + p4 submit -d \"Add another binary file with same content and spaces in path\" &&\n> +\n> + echo \"content 4 bin 26 bytes XX\" >file4.bin &&\n> + p4 add file4.bin &&\n> + p4 submit -d \"Add another binary file with different content\"\n> + )\n> +'\n> +\n> +test_expect_success 'Store files in Mock LFS based on size (>24 bytes)' '\n> + client_view \"//depot/... //client/...\" &&\n> + test_when_finished cleanup_git &&\n> + (\n> + cd \"$git\" &&\n> + git init . &&\n> + git config git-p4.useClientSpec true &&\n> + git config git-p4.largeFileSystem MockLFS &&\n> + git config git-p4.largeFileThreshold 24 &&\n> + git config git-p4.pushLargeFiles True &&\n> + git p4 clone --destination=\"$git\" //depot@all &&\n> +\n> + test_file_is_not_in_mock_lfs file1.txt \"content 1 txt 23 bytes\" &&\n> + test_file_is_in_mock_lfs file2.dat \"content 2-3 bin 25 bytes\" &&\n> + test_file_is_in_mock_lfs \"path with spaces/file3.bin\" \"content 2-3 bin 25 bytes\" &&\n> + test_file_is_in_mock_lfs file4.bin \"content 4 bin 26 bytes XX\" &&\n> +\n> + test_file_count_in_dir \".git/mock-storage/local\" 2 &&\n> + test_file_count_in_dir \".git/mock-storage/remote\" 2\n> + )\n> +'\n> +\n> +test_expect_success 'Store files in Mock LFS based on extension (dat)' '\n> + client_view \"//depot/... //client/...\" &&\n> + test_when_finished cleanup_git &&\n> + (\n> + cd \"$git\" &&\n> + git init . &&\n> + git config git-p4.useClientSpec true &&\n> + git config git-p4.largeFileSystem MockLFS &&\n> + git config git-p4.largeFileExtensions dat &&\n> + git config git-p4.pushLargeFiles True &&\n> + git p4 clone --destination=\"$git\" //depot@all &&\n> +\n> + test_file_is_not_in_mock_lfs file1.txt \"content 1 txt 23 bytes\" &&\n> + test_file_is_in_mock_lfs file2.dat \"content 2-3 bin 25 bytes\" &&\n> + test_file_is_not_in_mock_lfs \"path with spaces/file3.bin\" \"content 2-3 bin 25 bytes\" &&\n> + test_file_is_not_in_mock_lfs file4.bin \"content 4 bin 26 bytes XX\" &&\n> +\n> + test_file_count_in_dir \".git/mock-storage/local\" 1 &&\n> + test_file_count_in_dir \".git/mock-storage/remote\" 1\n> + )\n> +'\n> +\n> +test_expect_success 'Remove file from repo and store files in Mock LFS based on size (>24 bytes)' '\n> + client_view \"//depot/... //client/...\" &&\n> + (\n> + cd \"$cli\" &&\n> + p4 delete file4.bin &&\n> + p4 submit -d \"Remove file\"\n> + ) &&\n> +\n> + client_view \"//depot/... //client/...\" &&\n> + test_when_finished cleanup_git &&\n> + (\n> + cd \"$git\" &&\n> + git init . &&\n> + git config git-p4.useClientSpec true &&\n> + git config git-p4.largeFileSystem MockLFS &&\n> + git config git-p4.largeFileThreshold 24 &&\n> + git config git-p4.pushLargeFiles True &&\n> + git p4 clone --destination=\"$git\" //depot@all &&\n> +\n> + test_file_is_not_in_mock_lfs file1.txt \"content 1 txt 23 bytes\" &&\n> + test_file_is_in_mock_lfs file2.dat \"content 2-3 bin 25 bytes\" &&\n> + test_file_is_in_mock_lfs \"path with spaces/file3.bin\" \"content 2-3 bin 25 bytes\" &&\n> + test_file_is_deleted_in_mock_lfs file4.bin \"content 4 bin 26 bytes XX\" &&\n> +\n> + test_file_count_in_dir \".git/mock-storage/local\" 2 &&\n> + test_file_count_in_dir \".git/mock-storage/remote\" 2\n> + )\n> +'\n> +\n> +test_expect_success 'Run git p4 submit in repo configured with large file system' '\n> + client_view \"//depot/... //client/...\" &&\n> + test_when_finished cleanup_git &&\n> + (\n> + cd \"$git\" &&\n> + git init . &&\n> + git config git-p4.useClientSpec true &&\n> + git config git-p4.largeFileSystem MockLFS &&\n> + git config git-p4.largeFileThreshold 24 &&\n> + git config git-p4.pushLargeFiles True &&\n> + git p4 clone --destination=\"$git\" //depot@all &&\n> +\n> + test_must_fail git p4 submit\n> + )\n> +'\n> +\n> +test_expect_success 'kill p4d' '\n> + kill_p4d\n> +'\n> +\n> +test_done\n> --\n> 2.5.1\n>\n"}]}