{"thread":{"id":"40134","subject":"[PATCH v2] git-p4: fix faulty paths for case insensitive systems","startedAt":"2015-08-19T20:04:45Z","lastAt":"2015-08-21T08:10:46Z","messageCount":9,"participants":["larsxschneider@gmail.com","Torsten Bögershausen","Lars Schneider","Eric Sunshine","Junio C Hamano"],"isPatch":true,"patchVersion":2,"patchTotal":null},"messages":[{"id":"268348","messageId":"1440014686-63290-1-git-send-email-larsxschneider@gmail.com","threadId":"40134","inReplyTo":null,"subject":"[PATCH v2] git-p4: fix faulty paths for case insensitive systems","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-08-19T20:04:45Z","receivedAt":"2015-08-19T20:04:45Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider <larsxschneider@gmail.com>\n\nHi,\n\nas discussed with Luke in \"[PATCH] git-p4: fix faulty paths for case insensitive\nsystems\" I added a test case for my path fix. I also changed the trigger for the\nfix to the command line parameter \"--fix-paths\".\n\nCheers,\nLars\n\nLars Schneider (1):\n  git-p4: fix faulty paths for case insensitive systems\n\n git-p4.py                         | 83 +++++++++++++++++++++++++++++++++--\n t/t9821-git-p4-path-variations.sh | 91 +++++++++++++++++++++++++++++++++++++++\n 2 files changed, 170 insertions(+), 4 deletions(-)\n create mode 100755 t/t9821-git-p4-path-variations.sh\n\n--\n1.9.5 (Apple Git-50.3)\n"},{"id":"268349","messageId":"1440014686-63290-2-git-send-email-larsxschneider@gmail.com","threadId":"40134","inReplyTo":"1440014686-63290-1-git-send-email-larsxschneider@gmail.com","subject":"[PATCH v2] git-p4: fix faulty paths for case insensitive systems","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-08-19T20:04:46Z","receivedAt":"2015-08-19T20:04:46Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider <larsxschneider@gmail.com>\n\nPROBLEM:\nWe run P4 servers on Linux and P4 clients on Windows. For an unknown\nreason the file path for a number of files in P4 does not match the\ndirectory path with respect to case sensitivity.\n\nE.g. `p4 files` might return\n//depot/path/to/file1\n//depot/PATH/to/file2\n\nIf you use P4/P4V then these files end up in the same directory, e.g.\n//depot/path/to/file1\n//depot/path/to/file2\n\nIf you use git-p4 then all files not matching the correct file path\n(e.g. `file2`) will be ignored.\n\nSOLUTION:\nIdentify files that are different with respect to case sensitivity.\nIf there are any then run `p4 dirs` to build up a dictionary\ncontaining the \"correct\" cases for each path. It looks like P4\ninterprets \"correct\" here as the existing path of the first file in a\ndirectory. The path dictionary is used later on to fix all paths.\n\nThis is only applied if the parameter \"--fix-paths\" is passed to the\ngit-p4 clone command.\n\nSigned-off-by: Lars Schneider <larsxschneider@gmail.com>\n---\n git-p4.py                         | 83 +++++++++++++++++++++++++++++++++--\n t/t9821-git-p4-path-variations.sh | 91 +++++++++++++++++++++++++++++++++++++++\n 2 files changed, 170 insertions(+), 4 deletions(-)\n create mode 100755 t/t9821-git-p4-path-variations.sh\n\ndiff --git a/git-p4.py b/git-p4.py\nindex 073f87b..a21809d 100755\n--- a/git-p4.py\n+++ b/git-p4.py\n@@ -1931,7 +1931,7 @@ class View(object):\n                 (self.client_prefix, clientFile))\n         return clientFile[len(self.client_prefix):]\n\n-    def update_client_spec_path_cache(self, files):\n+    def update_client_spec_path_cache(self, files, fixPathCase = None):\n         \"\"\" Caching file paths by \"p4 where\" batch query \"\"\"\n\n         # List depot file paths exclude that already cached\n@@ -1950,6 +1950,8 @@ class View(object):\n             if \"unmap\" in res:\n                 # it will list all of them, but only one not unmap-ped\n                 continue\n+            if fixPathCase:\n+                res['depotFile'] = fixPathCase(res['depotFile'])\n             self.client_spec_path_cache[res['depotFile']] = self.convert_client_path(res[\"clientFile\"])\n\n         # not found files or unmap files set to \"\"\n@@ -1987,6 +1989,7 @@ class P4Sync(Command, P4UserMap):\n                                      help=\"Maximum number of changes to import\"),\n                 optparse.make_option(\"--changes-block-size\", dest=\"changes_block_size\", type=\"int\",\n                                      help=\"Internal block size to use when iteratively calling p4 changes\"),\n+                optparse.make_option(\"--fix-paths\", dest=\"fixPaths\", action=\"store_true\"),\n                 optparse.make_option(\"--keep-path\", dest=\"keepRepoPath\", action='store_true',\n                                      help=\"Keep entire BRANCH/DIR/SUBDIR prefix during import\"),\n                 optparse.make_option(\"--use-client-spec\", dest=\"useClientSpec\", action='store_true',\n@@ -2017,6 +2020,7 @@ class P4Sync(Command, P4UserMap):\n         self.maxChanges = \"\"\n         self.changes_block_size = None\n         self.keepRepoPath = False\n+        self.fixPaths = False\n         self.depotPaths = None\n         self.p4BranchesInGit = []\n         self.cloneExclude = []\n@@ -2049,7 +2053,8 @@ class P4Sync(Command, P4UserMap):\n         files = []\n         fnum = 0\n         while commit.has_key(\"depotFile%s\" % fnum):\n-            path =  commit[\"depotFile%s\" % fnum]\n+            path = commit[\"depotFile%s\" % fnum]\n+            path = self.fixPathCase(path)\n\n             if [p for p in self.cloneExclude\n                 if p4PathStartsWith(path, p)]:\n@@ -2113,7 +2118,9 @@ class P4Sync(Command, P4UserMap):\n         branches = {}\n         fnum = 0\n         while commit.has_key(\"depotFile%s\" % fnum):\n-            path =  commit[\"depotFile%s\" % fnum]\n+            path = commit[\"depotFile%s\" % fnum]\n+            path = self.fixPathCase(path)\n+\n             found = [p for p in self.depotPaths\n                      if p4PathStartsWith(path, p)]\n             if not found:\n@@ -2240,6 +2247,10 @@ class P4Sync(Command, P4UserMap):\n             if marshalled[\"code\"] == \"error\":\n                 if \"data\" in marshalled:\n                     err = marshalled[\"data\"].rstrip()\n+\n+        if \"depotFile\" in marshalled:\n+            marshalled['depotFile'] = self.fixPathCase(marshalled['depotFile'])\n+\n         if err:\n             f = None\n             if self.stream_have_file_info:\n@@ -2314,6 +2325,7 @@ class P4Sync(Command, P4UserMap):\n\n             # do the last chunk\n             if self.stream_file.has_key('depotFile'):\n+                self.stream_file['depotFile'] = self.fixPathCase(self.stream_file['depotFile'])\n                 self.streamOneP4File(self.stream_file, self.stream_contents)\n\n     def make_email(self, userid):\n@@ -2371,7 +2383,8 @@ class P4Sync(Command, P4UserMap):\n                 sys.stderr.write(\"Ignoring file outside of prefix: %s\\n\" % f['path'])\n\n         if self.clientSpecDirs:\n-            self.clientSpecDirs.update_client_spec_path_cache(files)\n+            self.clientSpecDirs.update_client_spec_path_cache(\n+                files, lambda x: self.fixPathCase(x))\n\n         self.gitStream.write(\"commit %s\\n\" % branch)\n #        gitStream.write(\"mark :%s\\n\" % details[\"change\"])\n@@ -2835,6 +2848,63 @@ class P4Sync(Command, P4UserMap):\n             print \"IO error with git fast-import. Is your git version recent enough?\"\n             print self.gitError.read()\n\n+    def fixPathCase(self, path):\n+        if self.caseCorrectedPaths:\n+            components = path.split('/')\n+            filename = components.pop()\n+            dirname = '/'.join(components).lower() + '/'\n+            if dirname in self.caseCorrectedPaths:\n+                path = self.caseCorrectedPaths[dirname] + filename\n+        return path\n+\n+    def generatePathCaseDict(self, depotPaths):\n+        # Query all files and generate a list of all used paths\n+        # e.g. this files list:\n+        # //depot/path/to/file1\n+        # //depot/PATH/to/file2\n+        #\n+        # result in this path list:\n+        # //depot/\n+        # //depot/PATH/\n+        # //depot/path/\n+        # //depot/PATH/to/\n+        # //depot/path/to/\n+        p4_paths = set()\n+        for p in depotPaths:\n+            for f in p4CmdList([\"files\", p+\"...\"]):\n+                components = f[\"depotFile\"].split('/')[0:-1]\n+                for i in range(3, len(components)+1):\n+                    p4_paths.add('/'.join(components[0:i]) + '/')\n+        p4_paths = sorted(list(p4_paths), key=len)\n+\n+        if len(p4_paths) > len(set([p.lower() for p in p4_paths])):\n+            print \"ATTENTION: File paths with different case variations detected. Fixing may take a while...\"\n+            found_variations = True\n+            while found_variations:\n+                for path in p4_paths:\n+                    found_variations = False\n+                    path_variations = [p for p in p4_paths if p.lower() == path.lower()]\n+\n+                    if len(path_variations) > 1:\n+                        print  \"%i different case variations for path '%s' detected.\" % (len(path_variations), path)\n+                        # If we detect path variations (e.g. //depot/path and //depot/PATH) then we query P4 to list\n+                        # the subdirectories of the parent (e.g //depot/*). P4 will return these subdirectories with\n+                        # the correct case.\n+                        parent_path = '/'.join(path.split('/')[0:-2])\n+                        case_ok_paths = [p[\"dir\"] + '/' for p in p4CmdList([\"dirs\", \"-D\", parent_path + '/*'])]\n+\n+                        # Replace all known paths with the case corrected path from P4 dirs command\n+                        for case_ok_path in case_ok_paths:\n+                            pattern = re.compile(\"^\" + re.escape(case_ok_path), re.IGNORECASE)\n+                            p4_paths = sorted(list(set([pattern.sub(case_ok_path, p) for p in p4_paths])), key=len)\n+\n+                        found_variations = True\n+                        break\n+            return dict((p.lower(), p) for p in p4_paths)\n+        else:\n+            if self.verbose:\n+                print \"All file paths have consistent case\"\n+            return None\n\n     def run(self, args):\n         self.depotPaths = []\n@@ -3006,6 +3076,11 @@ class P4Sync(Command, P4UserMap):\n\n         self.depotPaths = newPaths\n\n+        if self.fixPaths:\n+            self.caseCorrectedPaths = self.generatePathCaseDict(self.depotPaths)\n+        else:\n+            self.caseCorrectedPaths = None\n+\n         # --detect-branches may change this for each branch\n         self.branchPrefixes = self.depotPaths\n\ndiff --git a/t/t9821-git-p4-path-variations.sh b/t/t9821-git-p4-path-variations.sh\nnew file mode 100755\nindex 0000000..ede772f\n--- /dev/null\n+++ b/t/t9821-git-p4-path-variations.sh\n@@ -0,0 +1,91 @@\n+#!/bin/sh\n+\n+test_description='Clone repositories with path case variations'\n+\n+. ./lib-git-p4.sh\n+\n+test_expect_success 'start p4d with case folding enabled' '\n+\tstart_p4d -C1\n+'\n+\n+test_expect_success 'Create a repo with path case variations' '\n+\tclient_view \"//depot/... //client/...\" &&\n+\tcd \"$cli\" &&\n+\n+\tmkdir -p One/two &&\n+\t> One/two/File2.txt &&\n+\tp4 add One/two/File2.txt &&\n+\tp4 submit -d \"Add file2\" &&\n+\trm -rf One &&\n+\n+\tmkdir -p one/TWO &&\n+\t> one/TWO/file1.txt &&\n+\tp4 add one/TWO/file1.txt &&\n+\tp4 submit -d \"Add file1\" &&\n+\trm -rf one &&\n+\n+\tmkdir -p one/two &&\n+\t> one/two/file3.txt &&\n+\tp4 add one/two/file3.txt &&\n+\tp4 submit -d \"Add file3\" &&\n+\trm -rf one\n+'\n+\n+test_expect_success 'Clone the repo and WITHOUT path fixing' '\n+\tclient_view \"//depot/One/... //client/...\" &&\n+\tgit p4 clone --use-client-spec --destination=\"$git\" //depot/one &&\n+\ttest_when_finished cleanup_git &&\n+\t(\n+\t\tcd \"$git\" &&\n+\t\tfind . | grep two/File2.txt &&\n+\t\tgit ls-files > lines &&\n+\t\ttest_line_count = 1 lines\n+\t)\n+'\n+\n+test_expect_success 'Clone the repo WITH path fixing' '\n+\tclient_view \"//depot/One/... //client/...\" &&\n+\tgit p4 clone --fix-paths --use-client-spec --destination=\"$git\" //depot/one &&\n+\ttest_when_finished cleanup_git &&\n+\t(\n+\t\tcd \"$git\" &&\n+\t\tfind . | grep TWO/file1.txt &&\n+\t\tfind . | grep TWO/File2.txt &&\n+\t\tfind . | grep TWO/file3.txt &&\n+\t\tgit ls-files > lines &&\n+\t\ttest_line_count = 3 lines\n+\t)\n+'\n+\n+# It looks like P4 determines the path case based on the first file in\n+# lexicographical order. Please note the lower case \"two\" directory for all\n+# files triggered through the addition of \"File0.txt\".\n+test_expect_success 'Add a new file and clone the repo WITH path fixing' '\n+\tclient_view \"//depot/... //client/...\" &&\n+\tcd \"$cli\" &&\n+\n+\tmkdir -p One/two &&\n+\t> One/two/File0.txt &&\n+\tp4 add One/two/File0.txt &&\n+\tp4 submit -d \"Add file\" &&\n+\trm -rf One &&\n+\n+\tclient_view \"//depot/One/... //client/...\" &&\n+\tgit p4 clone --fix-paths --use-client-spec --destination=\"$git\" //depot/one &&\n+\ttest_when_finished cleanup_git &&\n+\t(\n+\t\tcd \"$git\" &&\n+\t\tfind . | grep two/File0.txt &&\n+\t\tfind . | grep two/file1.txt &&\n+\t\tfind . | grep two/File2.txt &&\n+\t\tfind . | grep two/file3.txt &&\n+\t\tgit ls-files > lines &&\n+\t\ttest_line_count = 4 lines\n+\t)\n+'\n+\n+test_expect_success 'kill p4d' '\n+\tkill_p4d\n+'\n+\n+test_done\n--\n1.9.5 (Apple Git-50.3)\n"},{"id":"268354","messageId":"55D55EC9.6050002@web.de","threadId":"40134","inReplyTo":"1440014686-63290-2-git-send-email-larsxschneider@gmail.com","subject":"Re: [PATCH v2] git-p4: fix faulty paths for case insensitive systems","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2015-08-20T04:59:53Z","receivedAt":"2015-08-20T04:59:53Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"Some nit-picking below:\nOn 08/19/2015 10:04 PM, larsxschneider@gmail.com wrote:\n> From: Lars Schneider <larsxschneider@gmail.com>\n>\n> PROBLEM:\n> We run P4 servers on Linux and P4 clients on Windows. For an unknown\n> reason the file path for a number of files in P4 does not match the\n> directory path with respect to case sensitivity.\n>\n> E.g. `p4 files` might return\n> //depot/path/to/file1\n> //depot/PATH/to/file2\n>\n> If you use P4/P4V then these files end up in the same directory, e.g.\n> //depot/path/to/file1\n> //depot/path/to/file2\n>\n> If you use git-p4 then all files not matching the correct file path\n> (e.g. `file2`) will be ignored.\n>\n> SOLUTION:\n> Identify files that are different with respect to case sensitivity.\nThis may be confusing:\nIt's the \"file names\" that differ, not the file content.\nAnd in the rest of the patch the term \"path\" is used.\nHow about this:\n\nIdentify path names that are different with respect to case sensitivity.\n\n\n> If there are any then run `p4 dirs` to build up a dictionary\n> containing the \"correct\" cases for each path. It looks like P4\n> interprets \"correct\" here as the existing path of the first file in a\n> directory. The path dictionary is used later on to fix all paths.\n>\n> This is only applied if the parameter \"--fix-paths\" is passed to the\n> git-p4 clone command.\nThe \"fix-path\" doesn't tell a user what exactly is fixed.\nSomething like \"fix-path-case\" may be more self-explaining,\nbut I would simply use \"--ignore-case\", which is more in line\nwith Git. (But this is debatable)\n> Signed-off-by: Lars Schneider <larsxschneider@gmail.com>\n> ---\n>   git-p4.py                         | 83 +++++++++++++++++++++++++++++++++--\n>   t/t9821-git-p4-path-variations.sh | 91 +++++++++++++++++++++++++++++++++++++++\n>   2 files changed, 170 insertions(+), 4 deletions(-)\n>   create mode 100755 t/t9821-git-p4-path-variations.sh\n>\n> diff --git a/git-p4.py b/git-p4.py\n> index 073f87b..a21809d 100755\n> --- a/git-p4.py\n> +++ b/git-p4.py\n> @@ -1931,7 +1931,7 @@ class View(object):\n>                   (self.client_prefix, clientFile))\n>           return clientFile[len(self.client_prefix):]\n>\n> -    def update_client_spec_path_cache(self, files):\n> +    def update_client_spec_path_cache(self, files, fixPathCase = None):\n>           \"\"\" Caching file paths by \"p4 where\" batch query \"\"\"\n>\n>           # List depot file paths exclude that already cached\n> @@ -1950,6 +1950,8 @@ class View(object):\n>               if \"unmap\" in res:\n>                   # it will list all of them, but only one not unmap-ped\n>                   continue\n> +            if fixPathCase:\n> +                res['depotFile'] = fixPathCase(res['depotFile'])\n>               self.client_spec_path_cache[res['depotFile']] = self.convert_client_path(res[\"clientFile\"])\n>\n>           # not found files or unmap files set to \"\"\n> @@ -1987,6 +1989,7 @@ class P4Sync(Command, P4UserMap):\n>                                        help=\"Maximum number of changes to import\"),\n>                   optparse.make_option(\"--changes-block-size\", dest=\"changes_block_size\", type=\"int\",\n>                                        help=\"Internal block size to use when iteratively calling p4 changes\"),\n> +                optparse.make_option(\"--fix-paths\", dest=\"fixPaths\", action=\"store_true\"),\n>                   optparse.make_option(\"--keep-path\", dest=\"keepRepoPath\", action='store_true',\n>                                        help=\"Keep entire BRANCH/DIR/SUBDIR prefix during import\"),\n>                   optparse.make_option(\"--use-client-spec\", dest=\"useClientSpec\", action='store_true',\n> @@ -2017,6 +2020,7 @@ class P4Sync(Command, P4UserMap):\n>           self.maxChanges = \"\"\n>           self.changes_block_size = None\n>           self.keepRepoPath = False\n> +        self.fixPaths = False\n>           self.depotPaths = None\n>           self.p4BranchesInGit = []\n>           self.cloneExclude = []\n> @@ -2049,7 +2053,8 @@ class P4Sync(Command, P4UserMap):\n>           files = []\n>           fnum = 0\n>           while commit.has_key(\"depotFile%s\" % fnum):\n> -            path =  commit[\"depotFile%s\" % fnum]\n> +            path = commit[\"depotFile%s\" % fnum]\n> +            path = self.fixPathCase(path)\n>\n>               if [p for p in self.cloneExclude\n>                   if p4PathStartsWith(path, p)]:\n> @@ -2113,7 +2118,9 @@ class P4Sync(Command, P4UserMap):\n>           branches = {}\n>           fnum = 0\n>           while commit.has_key(\"depotFile%s\" % fnum):\n> -            path =  commit[\"depotFile%s\" % fnum]\n> +            path = commit[\"depotFile%s\" % fnum]\n> +            path = self.fixPathCase(path)\n> +\n>               found = [p for p in self.depotPaths\n>                        if p4PathStartsWith(path, p)]\n>               if not found:\n> @@ -2240,6 +2247,10 @@ class P4Sync(Command, P4UserMap):\n>               if marshalled[\"code\"] == \"error\":\n>                   if \"data\" in marshalled:\n>                       err = marshalled[\"data\"].rstrip()\n> +\n> +        if \"depotFile\" in marshalled:\n> +            marshalled['depotFile'] = self.fixPathCase(marshalled['depotFile'])\n> +\n>           if err:\n>               f = None\n>               if self.stream_have_file_info:\n> @@ -2314,6 +2325,7 @@ class P4Sync(Command, P4UserMap):\n>\n>               # do the last chunk\n>               if self.stream_file.has_key('depotFile'):\n> +                self.stream_file['depotFile'] = self.fixPathCase(self.stream_file['depotFile'])\n>                   self.streamOneP4File(self.stream_file, self.stream_contents)\n>\n>       def make_email(self, userid):\n> @@ -2371,7 +2383,8 @@ class P4Sync(Command, P4UserMap):\n>                   sys.stderr.write(\"Ignoring file outside of prefix: %s\\n\" % f['path'])\n>\n>           if self.clientSpecDirs:\n> -            self.clientSpecDirs.update_client_spec_path_cache(files)\n> +            self.clientSpecDirs.update_client_spec_path_cache(\n> +                files, lambda x: self.fixPathCase(x))\n>\n>           self.gitStream.write(\"commit %s\\n\" % branch)\n>   #        gitStream.write(\"mark :%s\\n\" % details[\"change\"])\n> @@ -2835,6 +2848,63 @@ class P4Sync(Command, P4UserMap):\n>               print \"IO error with git fast-import. Is your git version recent enough?\"\n>               print self.gitError.read()\n>\n> +    def fixPathCase(self, path):\n> +        if self.caseCorrectedPaths:\n> +            components = path.split('/')\n> +            filename = components.pop()\n> +            dirname = '/'.join(components).lower() + '/'\n> +            if dirname in self.caseCorrectedPaths:\n> +                path = self.caseCorrectedPaths[dirname] + filename\n> +        return path\n> +\n> +    def generatePathCaseDict(self, depotPaths):\n> +        # Query all files and generate a list of all used paths\n> +        # e.g. this files list:\n> +        # //depot/path/to/file1\n> +        # //depot/PATH/to/file2\n> +        #\n> +        # result in this path list:\n> +        # //depot/\n> +        # //depot/PATH/\n> +        # //depot/path/\n> +        # //depot/PATH/to/\n> +        # //depot/path/to/\n> +        p4_paths = set()\n> +        for p in depotPaths:\n> +            for f in p4CmdList([\"files\", p+\"...\"]):\n> +                components = f[\"depotFile\"].split('/')[0:-1]\n> +                for i in range(3, len(components)+1):\n> +                    p4_paths.add('/'.join(components[0:i]) + '/')\n> +        p4_paths = sorted(list(p4_paths), key=len)\n> +\n> +        if len(p4_paths) > len(set([p.lower() for p in p4_paths])):\n> +            print \"ATTENTION: File paths with different case variations detected. Fixing may take a while...\"\n> +            found_variations = True\n> +            while found_variations:\n> +                for path in p4_paths:\n> +                    found_variations = False\n> +                    path_variations = [p for p in p4_paths if p.lower() == path.lower()]\n> +\n> +                    if len(path_variations) > 1:\n> +                        print  \"%i different case variations for path '%s' detected.\" % (len(path_variations), path)\n> +                        # If we detect path variations (e.g. //depot/path and //depot/PATH) then we query P4 to list\n> +                        # the subdirectories of the parent (e.g //depot/*). P4 will return these subdirectories with\n> +                        # the correct case.\n> +                        parent_path = '/'.join(path.split('/')[0:-2])\n> +                        case_ok_paths = [p[\"dir\"] + '/' for p in p4CmdList([\"dirs\", \"-D\", parent_path + '/*'])]\n> +\n> +                        # Replace all known paths with the case corrected path from P4 dirs command\n> +                        for case_ok_path in case_ok_paths:\n> +                            pattern = re.compile(\"^\" + re.escape(case_ok_path), re.IGNORECASE)\n> +                            p4_paths = sorted(list(set([pattern.sub(case_ok_path, p) for p in p4_paths])), key=len)\n> +\n> +                        found_variations = True\n> +                        break\n> +            return dict((p.lower(), p) for p in p4_paths)\n> +        else:\n> +            if self.verbose:\n> +                print \"All file paths have consistent case\"\n> +            return None\n>\n>       def run(self, args):\n>           self.depotPaths = []\n> @@ -3006,6 +3076,11 @@ class P4Sync(Command, P4UserMap):\n>\n>           self.depotPaths = newPaths\n>\n> +        if self.fixPaths:\n> +            self.caseCorrectedPaths = self.generatePathCaseDict(self.depotPaths)\n> +        else:\n> +            self.caseCorrectedPaths = None\n> +\n>           # --detect-branches may change this for each branch\n>           self.branchPrefixes = self.depotPaths\n>\n> diff --git a/t/t9821-git-p4-path-variations.sh b/t/t9821-git-p4-path-variations.sh\n> new file mode 100755\n> index 0000000..ede772f\n> --- /dev/null\n> +++ b/t/t9821-git-p4-path-variations.sh\n> @@ -0,0 +1,91 @@\n> +#!/bin/sh\n> +\n> +test_description='Clone repositories with path case variations'\n> +\n> +. ./lib-git-p4.sh\n> +\n> +test_expect_success 'start p4d with case folding enabled' '\n> +\tstart_p4d -C1\n> +'\n> +\n> +test_expect_success 'Create a repo with path case variations' '\n> +\tclient_view \"//depot/... //client/...\" &&\n> +\tcd \"$cli\" &&\n> +\n> +\tmkdir -p One/two &&\n> +\t> One/two/File2.txt &&\nMinor remark: To be more Git-style, please no space after '>':\n\n>One/two/File2.txt &&\n\n\n> +\tp4 add One/two/File2.txt &&\n> +\tp4 submit -d \"Add file2\" &&\n> +\trm -rf One &&\n> +\n> +\tmkdir -p one/TWO &&\n> +\t> one/TWO/file1.txt &&\nSame here\n> +\tp4 add one/TWO/file1.txt &&\n> +\tp4 submit -d \"Add file1\" &&\n> +\trm -rf one &&\n> +\n> +\tmkdir -p one/two &&\n> +\t> one/two/file3.txt &&\nand here\n> +\tp4 add one/two/file3.txt &&\n> +\tp4 submit -d \"Add file3\" &&\n> +\trm -rf one\n> +'\n> +\n> +test_expect_success 'Clone the repo and WITHOUT path fixing' '\n> +\tclient_view \"//depot/One/... //client/...\" &&\n> +\tgit p4 clone --use-client-spec --destination=\"$git\" //depot/one &&\n> +\ttest_when_finished cleanup_git &&\n> +\t(\n> +\t\tcd \"$git\" &&\nThe cd command should be in a subshell:\n\n     (\n         cd $git &&\n         test -f xxxx &&\n         test_this_and_that\n     )\n\n\nWriting shell scripts isn't easy,\nthere is some information in\nDocumentation/CodingGuidelines\nand\nt/README\n\n> +\t\tfind . | grep two/File2.txt &&\nShould we make sure that two/File2.txt exist?\nThen the \"find | grep\" feels like overkill.\nThe line\ntest -f  two/File2.txt &&\ncould do the same (or do I miss something ?)\n> +\t\tgit ls-files > lines &&\nand here\n> +\t\ttest_line_count = 1 lines\n> +\t)\n> +'\n> +\n> +test_expect_success 'Clone the repo WITH path fixing' '\n> +\tclient_view \"//depot/One/... //client/...\" &&\n> +\tgit p4 clone --fix-paths --use-client-spec --destination=\"$git\" //depot/one &&\n> +\ttest_when_finished cleanup_git &&\n> +\t(\n> +\t\tcd \"$git\" &&\n> +\t\tfind . | grep TWO/file1.txt &&\n> +\t\tfind . | grep TWO/File2.txt &&\n> +\t\tfind . | grep TWO/file3.txt &&\nNot sure about the find | grep here either.\n> +\t\tgit ls-files > lines &&\n> +\t\ttest_line_count = 3 lines\n> +\t)\n> +'\n> +\n> +# It looks like P4 determines the path case based on the first file in\n> +# lexicographical order. Please note the lower case \"two\" directory for all\n> +# files triggered through the addition of \"File0.txt\".\n> +test_expect_success 'Add a new file and clone the repo WITH path fixing' '\n> +\tclient_view \"//depot/... //client/...\" &&\n> +\tcd \"$cli\" &&\n> +\n> +\tmkdir -p One/two &&\n> +\t> One/two/File0.txt &&\n> +\tp4 add One/two/File0.txt &&\n> +\tp4 submit -d \"Add file\" &&\n> +\trm -rf One &&\n> +\n> +\tclient_view \"//depot/One/... //client/...\" &&\n> +\tgit p4 clone --fix-paths --use-client-spec --destination=\"$git\" //depot/one &&\n> +\ttest_when_finished cleanup_git &&\n> +\t(\n> +\t\tcd \"$git\" &&\n> +\t\tfind . | grep two/File0.txt &&\n> +\t\tfind . | grep two/file1.txt &&\n> +\t\tfind . | grep two/File2.txt &&\n> +\t\tfind . | grep two/file3.txt &&\n> +\t\tgit ls-files > lines &&\n> +\t\ttest_line_count = 4 lines\n> +\t)\n> +'\n> +\n> +test_expect_success 'kill p4d' '\n> +\tkill_p4d\n> +'\n> +\n> +test_done\n> --\n>\n"},{"id":"268355","messageId":"917E3751-4BC8-4034-9101-30C77426D81C@gmail.com","threadId":"40134","inReplyTo":"55D55EC9.6050002@web.de","subject":"Re: [PATCH v2] git-p4: fix faulty paths for case insensitive systems","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-08-20T07:16:39Z","receivedAt":"2015-08-20T07:16:39Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"Thanks for your feedback! See my answers below.\n\nOn 20 Aug 2015, at 06:59, Torsten Bögershausen <tboegi@web.de> wrote:\n\n> Some nit-picking below:\n> On 08/19/2015 10:04 PM, larsxschneider@gmail.com wrote:\n>> From: Lars Schneider <larsxschneider@gmail.com>\n>> \n>> PROBLEM:\n>> We run P4 servers on Linux and P4 clients on Windows. For an unknown\n>> reason the file path for a number of files in P4 does not match the\n>> directory path with respect to case sensitivity.\n>> \n>> E.g. `p4 files` might return\n>> //depot/path/to/file1\n>> //depot/PATH/to/file2\n>> \n>> If you use P4/P4V then these files end up in the same directory, e.g.\n>> //depot/path/to/file1\n>> //depot/path/to/file2\n>> \n>> If you use git-p4 then all files not matching the correct file path\n>> (e.g. `file2`) will be ignored.\n>> \n>> SOLUTION:\n>> Identify files that are different with respect to case sensitivity.\n> This may be confusing:\n> It's the \"file names\" that differ, not the file content.\n> And in the rest of the patch the term \"path\" is used.\n> How about this:\n> \n> Identify path names that are different with respect to case sensitivity.\nAgreed!\n\n> \n> \n>> If there are any then run `p4 dirs` to build up a dictionary\n>> containing the \"correct\" cases for each path. It looks like P4\n>> interprets \"correct\" here as the existing path of the first file in a\n>> directory. The path dictionary is used later on to fix all paths.\n>> \n>> This is only applied if the parameter \"--fix-paths\" is passed to the\n>> git-p4 clone command.\n> The \"fix-path\" doesn't tell a user what exactly is fixed.\n> Something like \"fix-path-case\" may be more self-explaining,\n> but I would simply use \"--ignore-case\", which is more in line\n> with Git. (But this is debatable)\nI agree but “—ignore-case” seems a bit broad to me. How about “—ignore-path-case”? Because it is really only about paths case, not filename case.\n\n\n>> Signed-off-by: Lars Schneider <larsxschneider@gmail.com>\n>> ---\n>>  git-p4.py                         | 83 +++++++++++++++++++++++++++++++++--\n>>  t/t9821-git-p4-path-variations.sh | 91 +++++++++++++++++++++++++++++++++++++++\n>>  2 files changed, 170 insertions(+), 4 deletions(-)\n>>  create mode 100755 t/t9821-git-p4-path-variations.sh\n>> \n>> diff --git a/git-p4.py b/git-p4.py\n>> index 073f87b..a21809d 100755\n>> --- a/git-p4.py\n>> +++ b/git-p4.py\n>> @@ -1931,7 +1931,7 @@ class View(object):\n>>                  (self.client_prefix, clientFile))\n>>          return clientFile[len(self.client_prefix):]\n>> \n>> -    def update_client_spec_path_cache(self, files):\n>> +    def update_client_spec_path_cache(self, files, fixPathCase = None):\n>>          \"\"\" Caching file paths by \"p4 where\" batch query \"\"\"\n>> \n>>          # List depot file paths exclude that already cached\n>> @@ -1950,6 +1950,8 @@ class View(object):\n>>              if \"unmap\" in res:\n>>                  # it will list all of them, but only one not unmap-ped\n>>                  continue\n>> +            if fixPathCase:\n>> +                res['depotFile'] = fixPathCase(res['depotFile'])\n>>              self.client_spec_path_cache[res['depotFile']] = self.convert_client_path(res[\"clientFile\"])\n>> \n>>          # not found files or unmap files set to \"\"\n>> @@ -1987,6 +1989,7 @@ class P4Sync(Command, P4UserMap):\n>>                                       help=\"Maximum number of changes to import\"),\n>>                  optparse.make_option(\"--changes-block-size\", dest=\"changes_block_size\", type=\"int\",\n>>                                       help=\"Internal block size to use when iteratively calling p4 changes\"),\n>> +                optparse.make_option(\"--fix-paths\", dest=\"fixPaths\", action=\"store_true\"),\n>>                  optparse.make_option(\"--keep-path\", dest=\"keepRepoPath\", action='store_true',\n>>                                       help=\"Keep entire BRANCH/DIR/SUBDIR prefix during import\"),\n>>                  optparse.make_option(\"--use-client-spec\", dest=\"useClientSpec\", action='store_true',\n>> @@ -2017,6 +2020,7 @@ class P4Sync(Command, P4UserMap):\n>>          self.maxChanges = \"\"\n>>          self.changes_block_size = None\n>>          self.keepRepoPath = False\n>> +        self.fixPaths = False\n>>          self.depotPaths = None\n>>          self.p4BranchesInGit = []\n>>          self.cloneExclude = []\n>> @@ -2049,7 +2053,8 @@ class P4Sync(Command, P4UserMap):\n>>          files = []\n>>          fnum = 0\n>>          while commit.has_key(\"depotFile%s\" % fnum):\n>> -            path =  commit[\"depotFile%s\" % fnum]\n>> +            path = commit[\"depotFile%s\" % fnum]\n>> +            path = self.fixPathCase(path)\n>> \n>>              if [p for p in self.cloneExclude\n>>                  if p4PathStartsWith(path, p)]:\n>> @@ -2113,7 +2118,9 @@ class P4Sync(Command, P4UserMap):\n>>          branches = {}\n>>          fnum = 0\n>>          while commit.has_key(\"depotFile%s\" % fnum):\n>> -            path =  commit[\"depotFile%s\" % fnum]\n>> +            path = commit[\"depotFile%s\" % fnum]\n>> +            path = self.fixPathCase(path)\n>> +\n>>              found = [p for p in self.depotPaths\n>>                       if p4PathStartsWith(path, p)]\n>>              if not found:\n>> @@ -2240,6 +2247,10 @@ class P4Sync(Command, P4UserMap):\n>>              if marshalled[\"code\"] == \"error\":\n>>                  if \"data\" in marshalled:\n>>                      err = marshalled[\"data\"].rstrip()\n>> +\n>> +        if \"depotFile\" in marshalled:\n>> +            marshalled['depotFile'] = self.fixPathCase(marshalled['depotFile'])\n>> +\n>>          if err:\n>>              f = None\n>>              if self.stream_have_file_info:\n>> @@ -2314,6 +2325,7 @@ class P4Sync(Command, P4UserMap):\n>> \n>>              # do the last chunk\n>>              if self.stream_file.has_key('depotFile'):\n>> +                self.stream_file['depotFile'] = self.fixPathCase(self.stream_file['depotFile'])\n>>                  self.streamOneP4File(self.stream_file, self.stream_contents)\n>> \n>>      def make_email(self, userid):\n>> @@ -2371,7 +2383,8 @@ class P4Sync(Command, P4UserMap):\n>>                  sys.stderr.write(\"Ignoring file outside of prefix: %s\\n\" % f['path'])\n>> \n>>          if self.clientSpecDirs:\n>> -            self.clientSpecDirs.update_client_spec_path_cache(files)\n>> +            self.clientSpecDirs.update_client_spec_path_cache(\n>> +                files, lambda x: self.fixPathCase(x))\n>> \n>>          self.gitStream.write(\"commit %s\\n\" % branch)\n>>  #        gitStream.write(\"mark :%s\\n\" % details[\"change\"])\n>> @@ -2835,6 +2848,63 @@ class P4Sync(Command, P4UserMap):\n>>              print \"IO error with git fast-import. Is your git version recent enough?\"\n>>              print self.gitError.read()\n>> \n>> +    def fixPathCase(self, path):\n>> +        if self.caseCorrectedPaths:\n>> +            components = path.split('/')\n>> +            filename = components.pop()\n>> +            dirname = '/'.join(components).lower() + '/'\n>> +            if dirname in self.caseCorrectedPaths:\n>> +                path = self.caseCorrectedPaths[dirname] + filename\n>> +        return path\n>> +\n>> +    def generatePathCaseDict(self, depotPaths):\n>> +        # Query all files and generate a list of all used paths\n>> +        # e.g. this files list:\n>> +        # //depot/path/to/file1\n>> +        # //depot/PATH/to/file2\n>> +        #\n>> +        # result in this path list:\n>> +        # //depot/\n>> +        # //depot/PATH/\n>> +        # //depot/path/\n>> +        # //depot/PATH/to/\n>> +        # //depot/path/to/\n>> +        p4_paths = set()\n>> +        for p in depotPaths:\n>> +            for f in p4CmdList([\"files\", p+\"...\"]):\n>> +                components = f[\"depotFile\"].split('/')[0:-1]\n>> +                for i in range(3, len(components)+1):\n>> +                    p4_paths.add('/'.join(components[0:i]) + '/')\n>> +        p4_paths = sorted(list(p4_paths), key=len)\n>> +\n>> +        if len(p4_paths) > len(set([p.lower() for p in p4_paths])):\n>> +            print \"ATTENTION: File paths with different case variations detected. Fixing may take a while...\"\n>> +            found_variations = True\n>> +            while found_variations:\n>> +                for path in p4_paths:\n>> +                    found_variations = False\n>> +                    path_variations = [p for p in p4_paths if p.lower() == path.lower()]\n>> +\n>> +                    if len(path_variations) > 1:\n>> +                        print  \"%i different case variations for path '%s' detected.\" % (len(path_variations), path)\n>> +                        # If we detect path variations (e.g. //depot/path and //depot/PATH) then we query P4 to list\n>> +                        # the subdirectories of the parent (e.g //depot/*). P4 will return these subdirectories with\n>> +                        # the correct case.\n>> +                        parent_path = '/'.join(path.split('/')[0:-2])\n>> +                        case_ok_paths = [p[\"dir\"] + '/' for p in p4CmdList([\"dirs\", \"-D\", parent_path + '/*'])]\n>> +\n>> +                        # Replace all known paths with the case corrected path from P4 dirs command\n>> +                        for case_ok_path in case_ok_paths:\n>> +                            pattern = re.compile(\"^\" + re.escape(case_ok_path), re.IGNORECASE)\n>> +                            p4_paths = sorted(list(set([pattern.sub(case_ok_path, p) for p in p4_paths])), key=len)\n>> +\n>> +                        found_variations = True\n>> +                        break\n>> +            return dict((p.lower(), p) for p in p4_paths)\n>> +        else:\n>> +            if self.verbose:\n>> +                print \"All file paths have consistent case\"\n>> +            return None\n>> \n>>      def run(self, args):\n>>          self.depotPaths = []\n>> @@ -3006,6 +3076,11 @@ class P4Sync(Command, P4UserMap):\n>> \n>>          self.depotPaths = newPaths\n>> \n>> +        if self.fixPaths:\n>> +            self.caseCorrectedPaths = self.generatePathCaseDict(self.depotPaths)\n>> +        else:\n>> +            self.caseCorrectedPaths = None\n>> +\n>>          # --detect-branches may change this for each branch\n>>          self.branchPrefixes = self.depotPaths\n>> \n>> diff --git a/t/t9821-git-p4-path-variations.sh b/t/t9821-git-p4-path-variations.sh\n>> new file mode 100755\n>> index 0000000..ede772f\n>> --- /dev/null\n>> +++ b/t/t9821-git-p4-path-variations.sh\n>> @@ -0,0 +1,91 @@\n>> +#!/bin/sh\n>> +\n>> +test_description='Clone repositories with path case variations'\n>> +\n>> +. ./lib-git-p4.sh\n>> +\n>> +test_expect_success 'start p4d with case folding enabled' '\n>> +\tstart_p4d -C1\n>> +'\n>> +\n>> +test_expect_success 'Create a repo with path case variations' '\n>> +\tclient_view \"//depot/... //client/...\" &&\n>> +\tcd \"$cli\" &&\n>> +\n>> +\tmkdir -p One/two &&\n>> +\t> One/two/File2.txt &&\n> Minor remark: To be more Git-style, please no space after '>’:\nOK\n\n>> One/two/File2.txt &&\n> \n> \n>> +\tp4 add One/two/File2.txt &&\n>> +\tp4 submit -d \"Add file2\" &&\n>> +\trm -rf One &&\n>> +\n>> +\tmkdir -p one/TWO &&\n>> +\t> one/TWO/file1.txt &&\n> Same here\nOK\n\n>> +\tp4 add one/TWO/file1.txt &&\n>> +\tp4 submit -d \"Add file1\" &&\n>> +\trm -rf one &&\n>> +\n>> +\tmkdir -p one/two &&\n>> +\t> one/two/file3.txt &&\n> and here\nOK\n\n>> +\tp4 add one/two/file3.txt &&\n>> +\tp4 submit -d \"Add file3\" &&\n>> +\trm -rf one\n>> +'\n>> +\n>> +test_expect_success 'Clone the repo and WITHOUT path fixing' '\n>> +\tclient_view \"//depot/One/... //client/...\" &&\n>> +\tgit p4 clone --use-client-spec --destination=\"$git\" //depot/one &&\n>> +\ttest_when_finished cleanup_git &&\n>> +\t(\n>> +\t\tcd \"$git\" &&\n> The cd command should be in a subshell:\n> \n>    (\n>        cd $git &&\n>        test -f xxxx &&\n>        test_this_and_that\n>    )\n> \n> \n> Writing shell scripts isn't easy,\n> there is some information in\n> Documentation/CodingGuidelines\n> and\n> t/README\nCan you try to rephrase your comment? I don’t get it. My “cd command” is already in parenthesis (== subshell), no?\n\n> \n>> +\t\tfind . | grep two/File2.txt &&\n> Should we make sure that two/File2.txt exist?\n> Then the \"find | grep\" feels like overkill.\n> The line\n> test -f  two/File2.txt &&\n> could do the same (or do I miss something ?)\nThat’s what I did first. However, I am running OS X with HFS in case-insensitive mode (the default). Consequently “test -f” doesn’t care about the case. That’s why I used “grep”.\n\n>> +\t\tgit ls-files > lines &&\n> and here\nI really want to make sure only one file ends up in the repo.\n\n>> +\t\ttest_line_count = 1 lines\n>> +\t)\n>> +'\n>> +\n>> +test_expect_success 'Clone the repo WITH path fixing' '\n>> +\tclient_view \"//depot/One/... //client/...\" &&\n>> +\tgit p4 clone --fix-paths --use-client-spec --destination=\"$git\" //depot/one &&\n>> +\ttest_when_finished cleanup_git &&\n>> +\t(\n>> +\t\tcd \"$git\" &&\n>> +\t\tfind . | grep TWO/file1.txt &&\n>> +\t\tfind . | grep TWO/File2.txt &&\n>> +\t\tfind . | grep TWO/file3.txt &&\n> Not sure about the find | grep here either.\nSee answers above.\n\n>> +\t\tgit ls-files > lines &&\n>> +\t\ttest_line_count = 3 lines\n>> +\t)\n>> +'\n>> +\n>> +# It looks like P4 determines the path case based on the first file in\n>> +# lexicographical order. Please note the lower case \"two\" directory for all\n>> +# files triggered through the addition of \"File0.txt\".\n>> +test_expect_success 'Add a new file and clone the repo WITH path fixing' '\n>> +\tclient_view \"//depot/... //client/...\" &&\n>> +\tcd \"$cli\" &&\n>> +\n>> +\tmkdir -p One/two &&\n>> +\t> One/two/File0.txt &&\n>> +\tp4 add One/two/File0.txt &&\n>> +\tp4 submit -d \"Add file\" &&\n>> +\trm -rf One &&\n>> +\n>> +\tclient_view \"//depot/One/... //client/...\" &&\n>> +\tgit p4 clone --fix-paths --use-client-spec --destination=\"$git\" //depot/one &&\n>> +\ttest_when_finished cleanup_git &&\n>> +\t(\n>> +\t\tcd \"$git\" &&\n>> +\t\tfind . | grep two/File0.txt &&\n>> +\t\tfind . | grep two/file1.txt &&\n>> +\t\tfind . | grep two/File2.txt &&\n>> +\t\tfind . | grep two/file3.txt &&\n>> +\t\tgit ls-files > lines &&\n>> +\t\ttest_line_count = 4 lines\n>> +\t)\n>> +'\n>> +\n>> +test_expect_success 'kill p4d' '\n>> +\tkill_p4d\n>> +'\n>> +\n>> +test_done\n>> --\n"},{"id":"268361","messageId":"CAPig+cQ8P8BDBGCNprN4GLtZbP0MHdj08LNw2WEJ87K5Ws1AHg@mail.gmail.com","threadId":"40134","inReplyTo":"917E3751-4BC8-4034-9101-30C77426D81C@gmail.com","subject":"Re: [PATCH v2] git-p4: fix faulty paths for case insensitive systems","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2015-08-20T13:17:20Z","receivedAt":"2015-08-20T13:17:20Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Thu, Aug 20, 2015 at 3:16 AM, Lars Schneider\n<larsxschneider@gmail.com> wrote:\n> On 20 Aug 2015, at 06:59, Torsten Bögershausen <tboegi@web.de> wrote:\n>> On 08/19/2015 10:04 PM, larsxschneider@gmail.com wrote:\n>>> From: Lars Schneider <larsxschneider@gmail.com>\n>>> +            find . | grep two/File2.txt &&\n>> Should we make sure that two/File2.txt exist?\n>> Then the \"find | grep\" feels like overkill.\n>> The line\n>> test -f  two/File2.txt &&\n>> could do the same (or do I miss something ?)\n> That’s what I did first. However, I am running OS X with HFS in\n> case-insensitive mode (the default). Consequently “test -f”\n> doesn’t care about the case. That’s why I used “grep”.\n\nThis explanation may deserves to be recorded as an in-code comment to\nmake the next person who modifies the test code aware of the issue.\n\n>>> +            git ls-files > lines &&\n>> and here\n> I really want to make sure only one file ends up in the repo.\n>\n>>> +            test_line_count = 1 lines\n\nA more idiomatic way (in Git tests) to check both the file case and\nthe file count would be:\n\n    cat >expect <<-\\EOF &&\n    two/File2.txt\n    EOF\n    git ls-files >actual &&\n    test_cmp expect actual\n\n>>> +    )\n>>> +'\n>>> +\n>>> +test_expect_success 'Clone the repo WITH path fixing' '\n>>> +    client_view \"//depot/One/... //client/...\" &&\n>>> +    git p4 clone --fix-paths --use-client-spec --destination=\"$git\" //depot/one &&\n>>> +    test_when_finished cleanup_git &&\n>>> +    (\n>>> +            cd \"$git\" &&\n>>> +            find . | grep TWO/file1.txt &&\n>>> +            find . | grep TWO/File2.txt &&\n>>> +            find . | grep TWO/file3.txt &&\n>> Not sure about the find | grep here either.\n> See answers above.\n>\n>>> +            git ls-files > lines &&\n>>> +            test_line_count = 3 lines\n\nLikewise:\n\n    cat >expect <<-\\EOF &&\n    TWO/File2.txt\n    TWO/file1.txt\n    TWO/file3.txt\n    EOF\n    git ls-files >actual &&\n    test_cmp expect actual\n\n>>> +    )\n>>> +'\n>>> +\n>>> +# It looks like P4 determines the path case based on the first file in\n>>> +# lexicographical order. Please note the lower case \"two\" directory for all\n>>> +# files triggered through the addition of \"File0.txt\".\n>>> +test_expect_success 'Add a new file and clone the repo WITH path fixing' '\n>>> +    client_view \"//depot/... //client/...\" &&\n>>> +    cd \"$cli\" &&\n>>> +\n>>> +    mkdir -p One/two &&\n>>> +    > One/two/File0.txt &&\n>>> +    p4 add One/two/File0.txt &&\n>>> +    p4 submit -d \"Add file\" &&\n>>> +    rm -rf One &&\n>>> +\n>>> +    client_view \"//depot/One/... //client/...\" &&\n>>> +    git p4 clone --fix-paths --use-client-spec --destination=\"$git\" //depot/one &&\n>>> +    test_when_finished cleanup_git &&\n>>> +    (\n>>> +            cd \"$git\" &&\n>>> +            find . | grep two/File0.txt &&\n>>> +            find . | grep two/file1.txt &&\n>>> +            find . | grep two/File2.txt &&\n>>> +            find . | grep two/file3.txt &&\n>>> +            git ls-files > lines &&\n>>> +            test_line_count = 4 lines\n\nAnd:\n\n    cat >expect <<-\\EOF &&\n    two/File0.txt\n    two/File2.txt\n    two/file1.txt\n    two/file3.txt\n    EOF\n    git ls-files >actual &&\n    test_cmp expect actual\n\n>>> +    )\n>>> +'\n>>> +\n>>> +test_expect_success 'kill p4d' '\n>>> +    kill_p4d\n>>> +'\n>>> +\n>>> +test_done\n>>> --\n"},{"id":"268372","messageId":"xmqq37ze7y4o.fsf@gitster.dls.corp.google.com","threadId":"40134","inReplyTo":"917E3751-4BC8-4034-9101-30C77426D81C@gmail.com","subject":"Re: [PATCH v2] git-p4: fix faulty paths for case insensitive systems","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-08-20T15:55:35Z","receivedAt":"2015-08-20T15:55:35Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Lars Schneider <larsxschneider@gmail.com> writes:\n\n>>> +\t\tfind . | grep TWO/file1.txt &&\n>>> +\t\tfind . | grep TWO/File2.txt &&\n>>> +\t\tfind . | grep TWO/file3.txt &&\n>> Not sure about the find | grep here either.\n> See answers above.\n\nThese are not very good tests; they will match \"OTWO/file1.txto\",\ntoo.  Besides, it is wasteful to run find three times.\n\n        find ?* -print | sort >actual &&\n        cat >expect <<-\\EOF &&\n        TWO/file1.txt\n        ...\n        EOF\n\ttest_cmp expect actual\n\nor something like that instead?\n"},{"id":"268384","messageId":"55D62445.5030402@web.de","threadId":"40134","inReplyTo":"917E3751-4BC8-4034-9101-30C77426D81C@gmail.com","subject":"Re: [PATCH v2] git-p4: fix faulty paths for case insensitive systems","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2015-08-20T19:02:29Z","receivedAt":"2015-08-20T19:02:29Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 2015-08-20 09.16, Lars Schneider wrote:\n> Thanks for your feedback! See my answers below.\n>> Identify path names that are different with respect to case sensitivity.\n> Agreed!\n> \n>>\n>>\n>>> If there are any then run `p4 dirs` to build up a dictionary\n>>> containing the \"correct\" cases for each path. It looks like P4\n>>> interprets \"correct\" here as the existing path of the first file in a\n>>> directory. The path dictionary is used later on to fix all paths.\n>>>\n>>> This is only applied if the parameter \"--fix-paths\" is passed to the\n>>> git-p4 clone command.\n>> The \"fix-path\" doesn't tell a user what exactly is fixed.\n>> Something like \"fix-path-case\" may be more self-explaining,\n>> but I would simply use \"--ignore-case\", which is more in line\n>> with Git. (But this is debatable)\n> I agree but “—ignore-case” seems a bit broad to me. How about “—ignore-path-case”? Because it is really only about paths case, not filename case.\nOK, better.\n\n\n>>> -    def update_client_spec_path_cache(self, files):\n>>> +    def update_client_spec_path_cache(self, files, fixPathCase = None):\nIf you want, you can change the name of the parameter into ignorePathCase,\nbut I'm not sure...\n\n\n\n> Can you try to rephrase your comment? I don’t get it. My “cd command” is already in parenthesis (== subshell), no?\nI may have commented on the wrong TC :-(\n\nThis is the one:\n+test_expect_success 'Create a repo with path case variations' '\n+\tclient_view \"//depot/... //client/...\" &&\n+\tcd \"$cli\" &&\n+\n+\tmkdir -p One/two &&\n+\t[snip]\n+'\n\nwhich should be like this:\n+test_expect_success 'Create a repo with path case variations' '\n+ client_view \"//depot/... //client/...\" &&\n+ (\n+\tcd \"$cli\" &&\n+\tmkdir -p One/two &&\n+ \t[snip]\n+ )\n+'\n"},{"id":"268405","messageId":"C80E3141-C247-439B-97B5-562881C127F3@gmail.com","threadId":"40134","inReplyTo":"xmqq37ze7y4o.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH v2] git-p4: fix faulty paths for case insensitive systems","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-08-21T08:08:09Z","receivedAt":"2015-08-21T08:08:09Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"@Luke, Torsten, Eric, Junio:\nThanks for the great feedback. I incorporated everything into \"[PATCH v3] git-p4: fix faulty paths for case insensitive systems”.\nIs this the correct way? I have never worked with the email-patch-files before :-)\n\n- Lars\n\nOn 20 Aug 2015, at 17:55, Junio C Hamano <gitster@pobox.com> wrote:\n\n> Lars Schneider <larsxschneider@gmail.com> writes:\n> \n>>>> +\t\tfind . | grep TWO/file1.txt &&\n>>>> +\t\tfind . | grep TWO/File2.txt &&\n>>>> +\t\tfind . | grep TWO/file3.txt &&\n>>> Not sure about the find | grep here either.\n>> See answers above.\n> \n> These are not very good tests; they will match \"OTWO/file1.txto\",\n> too.  Besides, it is wasteful to run find three times.\n> \n>        find ?* -print | sort >actual &&\n>        cat >expect <<-\\EOF &&\n>        TWO/file1.txt\n>        ...\n>        EOF\n> \ttest_cmp expect actual\n> \n> or something like that instead?\n"},{"id":"268406","messageId":"CAPig+cQjNHB-VyNvWn2rmD6H6p3mOg2SQZCBfh+0sN=yU7UVpg@mail.gmail.com","threadId":"40134","inReplyTo":"C80E3141-C247-439B-97B5-562881C127F3@gmail.com","subject":"Re: [PATCH v2] git-p4: fix faulty paths for case insensitive systems","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2015-08-21T08:10:46Z","receivedAt":"2015-08-21T08:10:46Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Aug 21, 2015 at 4:08 AM, Lars Schneider\n<larsxschneider@gmail.com> wrote:\n> @Luke, Torsten, Eric, Junio:\n> Thanks for the great feedback. I incorporated everything into\n> \"[PATCH v3] git-p4: fix faulty paths for case insensitive systems”.\n> Is this the correct way? I have never worked with the email-patch-files before :-)\n\nI'm not a p4 user nor have I any knowledge of it, however, with\nregards to the issues my v2 comments touched upon, your v3 changes\nlook fine.\n"}]}